Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 9849 - mounted FS access/merge failures (baselayout)
Summary: mounted FS access/merge failures (baselayout)
Status: RESOLVED DUPLICATE of bug 23851
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] baselayout (show other bugs)
Hardware: x86 All
: High critical (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
: 11192 11939 12825 13331 13720 15264 18561 20334 23750 24806 27721 27865 28231 29043 29629 29670 29867 29991 30939 34970 38259 40044 40463 41957 42166 43037 43058 43067 43068 43090 43093 44321 44778 46689 47342 (view as bug list)
Depends on:
Blocks:
 
Reported: 2002-10-28 16:30 UTC by jochem prins
Modified: 2005-10-07 15:37 UTC (History)
31 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
changed some keepdir's to keepdir_mount (baselayout-1.8.6.10-r1-mount.patch,1.29 KB, patch)
2003-09-21 17:02 UTC, Sascha Silbe
Details | Diff
fix for devfs-using installers (baselayout-1.8.6.10-r1-devfs-installer.patch,453 bytes, patch)
2003-10-02 12:35 UTC, Sascha Silbe
Details | Diff
Portage-2.0.50-r4 patch to ignore failed directory or .keep file creation (portage-2.0.50-r4-keep.patch,3.28 KB, patch)
2004-04-07 21:26 UTC, Jason Stubbs (RETIRED)
Details | Diff
patch to die AFTER package is merged (portage-2.0.50-r4-keep.patch,4.20 KB, patch)
2004-04-07 22:40 UTC, Jason Stubbs (RETIRED)
Details | Diff
Same but supports new ebuild var NONCRITICAL_KEEPDIRS (portage-2.0.50-r4-keep.patch,4.63 KB, patch)
2004-04-08 19:10 UTC, Jason Stubbs (RETIRED)
Details | Diff
Simpler patch that keeps current behaviour except dying when .keep cannot be written (portage-2.0.50-r4-keep2.patch,1.71 KB, patch)
2004-04-11 18:26 UTC, Jason Stubbs (RETIRED)
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description jochem prins 2002-10-28 16:30:33 UTC
ebuild: baselayout-1.8.4.1(latest)
gcc: 2.95-r7
arch: ~x86
portage: 2.0.43
When i attempted to update my system, the ebuild failed due to the fact that i had mounted /mnt/cdrom for my cdrom. 
This problem can probably be avoided by a mount-check script, for mounted devices in /mnt.
Comment 1 Sascha Silbe 2002-10-29 09:48:02 UTC
Same problem here because /home is an NFS mount:

[...]
>>> /var/lib/misc/.keep

!!! Cannot write to '//home'.
!!! Please check permissions and directories for broken symlinks.
!!! You may start the merge process again by using ebuild:
!!! ebuild /usr/portage/sys-apps/baselayout/baselayout-1.8.4.1.ebuild merge
!!! And finish by running this: env-update

cube root # mount|grep /home
sphere:/home on /home type nfs (rw,noatime,addr=192.168.1.3)
cube root # 


Comment 2 Martin Schlemmer (RETIRED) gentoo-dev 2002-10-29 11:51:13 UTC
Nick .. not sure with this one, as it is baselayout's fault that it creates
these dirs, but then portage that actually fails when merging to /.
Comment 3 Nicholas Jones (RETIRED) gentoo-dev 2002-10-29 18:53:01 UTC
Ok. I'll take this one for a future release... It's a portage thing.
I guess I can try to move the access check after a conditional...


Do you have root squashing in effect for the NFS issue?
(root_squash is default) That'd kill the NFS issue.

Other than that... there really isn't a reason to have anything
mounted except maybe a distfiles CD.
Comment 4 John Steele Scott 2002-11-05 21:56:41 UTC
I am getting the same error as Sascha, and yes, it would be because my NFS
server squashes root. The only way I see around this (apart from changing the
NFS server settings, which not everyone can do) is to unmount /home, install the
baselayout, and then mount /home from NFS again. I'll give that a shot sometime,
but it would be better if it weren't necessary.
Comment 5 Sascha Silbe 2002-11-13 17:16:52 UTC
Yes, I do have root_squash enabled (you cannot convince me to change that, that would be a security hazard) and that's causing it to fail. 
I really don't like having to put the machine into single-user just to update sys-apps/baselayout, especially not if it gets changed regularly (as it is currently).
How about checking (and possibly creating) /home in postinstall for a temporary solution? Especially /home is a network mount on many systems.
Comment 6 SpanKY gentoo-dev 2002-11-25 10:03:44 UTC
*** Bug 11192 has been marked as a duplicate of this bug. ***
Comment 7 Michael Cummings (RETIRED) gentoo-dev 2002-11-25 10:26:44 UTC
Just to add an additional 2 cents worth - the problem also crops up if you have 
a cd mounted (baselayout wants to write a .keep file to /mnt/cdrom, which 
happens to be ro at the time).
Comment 8 Michael Cummings (RETIRED) gentoo-dev 2002-11-25 11:25:58 UTC
I'm an idiot - thanks to SpanKy for pointing out the opening bug comment.
Comment 9 Sascha Silbe 2002-11-25 16:12:40 UTC
It got even worse: After an unsuccessful update attempt (i.e. after an emerge -u system), my system could not boot properly anymore (thus it failed to get up after a power outage). I had to boot it in single user mode (which requires physical access, so it cannot be done remotely) and remerge baselayout.
As you see, it breaks running systems, so please raise the severity to critical and the priority to P1. And accepting the bug (instead of leaving it as new) would be fine, too.

Unfortunately, I don't have a copy of the error messages (too much to write down, no working system to cut&paste them). The most important problem was that /mnt/.init.d did not get mounted, so the whole rc-scripts system failed utterly.

Comment 10 SpanKY gentoo-dev 2002-12-11 13:09:19 UTC
*** Bug 11939 has been marked as a duplicate of this bug. ***
Comment 11 jochem prins 2002-12-11 13:25:10 UTC
Decided to raised severity & priority to critical, since it seems to fail for a
number of NFS users. Personally i didn't experience this bug anymore with my
current setup. Do note though, that i don't use NFS, thus i cannot verify if the
bug is fixed for NFS users.

my setup:
gcc: 2.95-r7
arch: ~x86
portage: 2.0.45-r3
baselayout: 1.8.5.5

Comment 12 Eric Thibodeau 2002-12-18 14:46:42 UTC
I am experiencing the same problem with /mnt/floppy and /mnt/cdrom. This problem popped up after I added automout capabilities to my kernel. If I got all of this correctly, the baselayour script writes a .keep in all of these /mnt directories, hence triggering the automount mechanism. You can bypass the /mnt/floppy write error by inserting one in the drive... but have fun writing to the cdrom though! 
Comment 13 SpanKY gentoo-dev 2002-12-28 19:46:38 UTC
*** Bug 12825 has been marked as a duplicate of this bug. ***
Comment 14 Martin Schlemmer (RETIRED) gentoo-dev 2003-01-07 13:15:43 UTC
*** Bug 13331 has been marked as a duplicate of this bug. ***
Comment 15 Martin Schlemmer (RETIRED) gentoo-dev 2003-01-07 13:16:42 UTC
Nick, status on this ?
Comment 16 Nicholas Jones (RETIRED) gentoo-dev 2003-01-07 13:43:23 UTC
Nothing I can really do about it... It'd be one heck of a performance hit to
check every filename and guess if it can be merged or not...

It'd be best if baselayout would be kind and check for ro mounts and
just avoid dropping the .keep files in those dirs.
Comment 17 Martin Schlemmer (RETIRED) gentoo-dev 2003-01-07 16:20:35 UTC
Ok, Ill try to get some sort of fix into the baselayout ebuild then.
Comment 18 SpanKY gentoo-dev 2003-01-12 11:31:42 UTC
*** Bug 13720 has been marked as a duplicate of this bug. ***
Comment 19 Martin Schlemmer (RETIRED) gentoo-dev 2003-02-09 04:28:40 UTC
*** Bug 15264 has been marked as a duplicate of this bug. ***
Comment 20 Norberto Bensa 2003-03-17 01:01:22 UTC
Just to add to this bug report I get: 
 
!!! Cannot write to '/usr/portage'. 
!!! Please check permissions and directories for broken symlinks. 
!!! You may start the merge process again by using ebuild: 
!!! ebuild /usr/portage/sys-apps/baselayout/baselayout-1.8.6.4.ebuild merge 
!!! And finish by running this: env-update 
 
/usr/portage is NFS and read-only. The ebuild wants to write a .keep file in there and fails. 
 
Thanks for reading, 
Norberto 
Comment 21 Martin Schlemmer (RETIRED) gentoo-dev 2003-03-31 17:37:18 UTC
*** Bug 18561 has been marked as a duplicate of this bug. ***
Comment 22 Martin Schlemmer (RETIRED) gentoo-dev 2003-05-04 03:05:42 UTC
*** Bug 20334 has been marked as a duplicate of this bug. ***
Comment 23 SpanKY gentoo-dev 2003-06-30 14:52:47 UTC
*** Bug 23750 has been marked as a duplicate of this bug. ***
Comment 24 SpanKY gentoo-dev 2003-07-19 13:41:25 UTC
*** Bug 24806 has been marked as a duplicate of this bug. ***
Comment 25 SpanKY gentoo-dev 2003-07-29 17:51:06 UTC
*** Bug 25542 has been marked as a duplicate of this bug. ***
Comment 26 SpanKY gentoo-dev 2003-09-02 05:00:19 UTC
*** Bug 27721 has been marked as a duplicate of this bug. ***
Comment 27 SpanKY gentoo-dev 2003-09-03 10:57:11 UTC
*** Bug 27865 has been marked as a duplicate of this bug. ***
Comment 28 SpanKY gentoo-dev 2003-09-08 18:38:06 UTC
*** Bug 28231 has been marked as a duplicate of this bug. ***
Comment 29 SpanKY gentoo-dev 2003-09-18 05:09:07 UTC
*** Bug 29043 has been marked as a duplicate of this bug. ***
Comment 30 Sascha Silbe 2003-09-21 17:02:42 UTC
Created attachment 18106 [details, diff]
changed some keepdir's to keepdir_mount

With this patch, I can update baselayout cleanly on my system.
Is there any reason not to replace keepdir with keepdir_mount completely?
Comment 31 SpanKY gentoo-dev 2003-09-26 00:15:24 UTC
*** Bug 29629 has been marked as a duplicate of this bug. ***
Comment 32 SpanKY gentoo-dev 2003-09-26 18:09:09 UTC
*** Bug 29670 has been marked as a duplicate of this bug. ***
Comment 33 SpanKY gentoo-dev 2003-09-28 21:52:04 UTC
*** Bug 29867 has been marked as a duplicate of this bug. ***
Comment 34 Sascha Silbe 2003-10-02 12:35:50 UTC
Created attachment 18614 [details, diff]
fix for devfs-using installers

This patch fixes problems with baselayout during install if /dev is a mounted
devfs file system (which is needed for ATA RAID systems).
Should fix Bug #29867 ("bootstrap fails with ErrorCode 13 Access Denied when
following ataraid directions").
Comment 35 Martin Schlemmer (RETIRED) gentoo-dev 2003-10-11 05:11:22 UTC
*** Bug 29991 has been marked as a duplicate of this bug. ***
Comment 36 Martin Schlemmer (RETIRED) gentoo-dev 2003-10-11 05:15:31 UTC
Comment #34:

> This patch fixes problems with baselayout during install if /dev is a mounted
> devfs file system (which is needed for ATA RAID systems).
> Should fix Bug #29867 ("bootstrap fails with ErrorCode 13 Access Denied
when
> following ataraid directions").

Good catch - I'll add it soonish ...
Comment 37 SpanKY gentoo-dev 2003-10-11 18:21:13 UTC
*** Bug 30939 has been marked as a duplicate of this bug. ***
Comment 38 Q 2003-10-11 18:35:43 UTC
Could someone please upgrade 'soonish' to urgently? I can't get passed bootstrap
becuase it wants to move .keep to /dev/.keep (or similar, the full ouput
has been moved to one of the duplicates above. It is quite simply preventing
Gentoo from installing, which I imagine is a pretty serious bug.

Q
Comment 39 Paul Belt 2003-10-13 09:02:50 UTC
!!! copy /var/tmp/portage/baselayout-1.8.6.10-r1/image/home/.keep -> /home/.keep
failed.
!!! [Errno 2] No such file or directory: '/home/.keep#new'

automount(pid12932) on /home type autofs (rw,fd=5,pgrp=12932,minproto=2,maxproto=3)

turn off automount and it merges fine.
Comment 40 Q 2003-10-13 09:59:46 UTC
How do I do that please? If this is the answer then thank you very much.
I hope the devs take this into account in the currently avaiable tarballs...

Q
Comment 41 Cal Evans 2003-10-14 07:28:06 UTC
Has any progress been made on this?

I'm getting the same error but the file is /dev/.keep Can't move past this
error. Plese help.

I'm doing a stage 1 install on a laptop.
Comment 42 Mauricio L. Pilla (RETIRED) gentoo-dev 2003-10-14 07:33:42 UTC
Have you tried Sascha Silbe's fix (available as fix_for_devfs-using_installers)
?
Comment 43 Cal Evans 2003-10-14 07:45:33 UTC
Yes, I've applied it and am now testing it.

When will this patch be rolled into baselayout though? (assuming it works,
and from looking at it, it should.)

Can anyone explain why mkdir /dev/.keep works but touch /dev/.keep does not?
(I'm jsut curious here)

=C=
Comment 44 Q 2003-10-14 09:53:04 UTC
As I have said, it would be cool if someone would acutally say 'how' to turn
automount off, if this is indeed the cause of this bug. I have no clue how
to do this - and I fear anyone searching through gentoo bugs may also bw
equally bemused.

It is very kind of the above individual to point out this is what is required,
but it cannot trully be considered a resolution until someone actually states
how this should be done.

Please can someone supply this answer? I can only asume that it will serve
at least to make the bugs database a fraction more coherant than is currently
the case.

Any input in this regard is welcome.

Q
Comment 45 Mauricio L. Pilla (RETIRED) gentoo-dev 2003-10-14 10:13:50 UTC
In my system, I have a autofs script in /etc/init.d/ which has been installed
by package net-fs/autofs. I just have to issue a

/etc/init.d/autofs stop

To stop it. Don't know if it the case for you. But anyway, you can be a naughty
boy and kill the automount process 

killall -9 automount

I think that it will work for you... But I haven't tested (only the first
method) and thus I cannot say if it will bork your system or not. Should
not IMO
Comment 46 Q 2003-10-14 12:42:25 UTC
Quote:>In my system, I have a autofs script in /etc/init.d/ which has been
>installed by package net-fs/autofs. I just have to issue a

>/etc/init.d/autofs stop

>To stop it. Don't know if it the case for you. But anyway, you can be a
>naughty boy and kill the automount process 

>killall -9 automount

No thanks for that but it didn't work. I did /etc/init.d/autofs stop and
all that happend was I got an error message saying "No such file file or
directory". When I tried to do killall -9 automount I got an message saying
automount: no processes killed.

What does all this mean and how can I stop this setvice? Is it even started?
If so, how can I tell?
Comment 47 Olav Kolbu 2003-10-15 06:59:44 UTC
Please everyone note that this isn't a /dev issue, or a /mnt issue, or a
/home issue, or an automount issue, so there is no real point in trying to
fix those with chewing gum and a piece of string. This is about baselayout
writing to directories it is not allowed to write to, for one reason or another.


As most people in a semi-large scale environment, I mount /home from a fileserver
and certainly not as root. So it fails for me unless I unmount /home and
try again. Pain in the neck. 

A more generic way of avoiding the problem should be thought of, even if
it just means having a make.conf option with a shortlist of directories to
avoid (both when writing to, when trying to remove empty dirs and any other
access). Having emerge prune according to this list and give an appropriate
warning shouldn't be both easy to implement and fast. The option could be
empty by default, thus all dirs will be created upon installation of the
machine in question. However, the user is subsequently allowed to add say
/home, /whatnot there if needed. Appropriate warnings against putting / in
the list is probably a good idea. :-)



Comment 48 Sascha Silbe 2003-10-21 03:39:08 UTC
@Cal: Did my patch work for you?

Comment 49 Cal Evans 2003-10-27 17:41:31 UTC
Yes, it did. When can we expect it to be incorporated into the main codebase?

=C=
Comment 50 Martin Schlemmer (RETIRED) gentoo-dev 2003-10-29 12:08:19 UTC
Its in .11.
Comment 51 SpanKY gentoo-dev 2003-12-03 10:02:33 UTC
*** Bug 34970 has been marked as a duplicate of this bug. ***
Comment 52 wal mart 2003-12-05 08:43:09 UTC
per bug 34970 its still a problem in .12
Comment 53 SpanKY gentoo-dev 2004-01-15 08:09:33 UTC
*** Bug 38259 has been marked as a duplicate of this bug. ***
Comment 54 SpanKY gentoo-dev 2004-02-01 08:39:37 UTC
*** Bug 40044 has been marked as a duplicate of this bug. ***
Comment 55 SpanKY gentoo-dev 2004-02-05 19:47:36 UTC
*** Bug 40463 has been marked as a duplicate of this bug. ***
Comment 56 Jeremy Huddleston (RETIRED) gentoo-dev 2004-02-08 17:39:26 UTC
The problem is still in 1.8.6.13...

>>> /mnt/.keep
>>> /mnt/cdrom/
>>> /mnt/cdrom/.keep
--- /mnt/floppy/
!!! copy /usr/tmp/portage/baselayout-1.8.6.13/image/mnt/floppy/.keep -> /mnt/floppy/.keep failed.
!!! [Errno 123] No medium found: '/mnt/floppy/.keep#new'
Comment 57 Troy Dack 2004-02-09 04:33:40 UTC
Baselayout still fails to merge for me in 1.8.6.13 because I have /home mounted over nfs with root_squash enabled.

Changing
	keepdir /home
to
	keepdir_mount /home

resolves this.
Comment 58 Ernst Bachmann 2004-02-09 23:40:16 UTC
I wonder why "keepdir" is still in use at all...
If portage is unable to add a ".keep" file to that directory now, it will be unable to delete that directory later anyways, so there is no danger of "accidently" deleting that directory during an unmerge...
Why not rename "keepdir_mount" to "keepdir" and drop the old keepdir completely?
Comment 59 Ernst Bachmann 2004-02-10 00:03:08 UTC
Another "brute force" fix could be the following:

mount --bind / /somewhere/root
mount --bind /usr /somewhere/root/usr
etc. (for each locally rw mounted fs in fstab)

now you have a complete "shadow" tree of your system in /somewhere/root, without all those nfs/autofs/whatever mounts that could cause trouble.
now merge into /somewhere/root, instead of /, and everything should work fine...

As a side effect, with this method you're even able to access those parts of the root filesystem that otherwise would be hidden by mounts, e.g. writing /somewhere/root/mnt/floppy/.keep would actually add the keep file to the floppy mountpoint, not on the floppy itself if one is mounted.
Comment 60 Neil Bingham 2004-02-10 06:00:10 UTC
When upgrading to 1.8.6.13 this morning I got this:

--- /home/
!!! copy /var/tmp/portage/baselayout-1.8.6.13/image/home/.keep -> /home/.keep failed.
!!! [Errno 13] Permission denied: '/home/.keep#new'

/home/ is under the control of autofs on my system as the workstation is bound into a NIS domain.  Stopping autofs to do the install sorted it, but that isn't exactly a good solution for a server that has active mounts.
Comment 61 Sascha Silbe 2004-02-11 02:48:51 UTC
Another baselayout update, another failure:

!!! Cannot write to '/home'.
!!! Please check permissions and directories for broken symlinks.
!!! You may start the merge process again by using ebuild:
!!! ebuild /usr/portage/sys-apps/baselayout/baselayout-1.8.6.13.ebuild merge
!!! And finish by running this: env-update


What is holding back my bugfix? I really hate having to apply local patches to base system packages for a regular system.

Comment 62 SpanKY gentoo-dev 2004-02-18 23:05:45 UTC
*** Bug 41957 has been marked as a duplicate of this bug. ***
Comment 63 SpanKY gentoo-dev 2004-02-20 12:41:47 UTC
*** Bug 42166 has been marked as a duplicate of this bug. ***
Comment 64 SpanKY gentoo-dev 2004-02-26 17:03:34 UTC
*** Bug 43037 has been marked as a duplicate of this bug. ***
Comment 65 SpanKY gentoo-dev 2004-02-26 23:03:13 UTC
*** Bug 43058 has been marked as a duplicate of this bug. ***
Comment 66 Tal Peer (RETIRED) gentoo-dev 2004-02-27 00:37:24 UTC
When i emerge baselayout-1.8.6.13-r1 i get this in the merge phase:
--- /dev/
--- /dev/pts/
!!! copy /var/tmp/portage/baselayout-1.8.6.13-r1/image/dev/pts/.keep -> /dev/pts/.keep failed.
!!! [Errno 13] Permission denied: '/dev/pts/.keep#new'

The machine uses devfs (and devfsd running) on kernel 2.4.24.
Comment 67 Phil Richards 2004-02-27 00:45:04 UTC
sys-apps/baselayout-1.8.6.13 works,
sys-apps/baselayout-1.8.6.13-r1 fails
(/dev/pts as per comment #66)
Comment 68 Martin Holzer (RETIRED) gentoo-dev 2004-02-27 01:25:28 UTC
*** Bug 43067 has been marked as a duplicate of this bug. ***
Comment 69 Martin Holzer (RETIRED) gentoo-dev 2004-02-27 01:32:58 UTC
*** Bug 43068 has been marked as a duplicate of this bug. ***
Comment 70 Martin Holzer (RETIRED) gentoo-dev 2004-02-27 05:36:23 UTC
*** Bug 43090 has been marked as a duplicate of this bug. ***
Comment 71 Andreas Vinsander 2004-02-27 06:56:35 UTC
Having problem emerging baselayout-1.8.6.13-r1 here as well.
(Running a pac-sources kernel with devfs.)

Looking at baselayout-1.8.6.13-r1.ebuild I noticed the following:

<code>
if [ "${altmerge}" -eq "1" ]
        then
                # rootfs and devfs
                #dosym ../../sbin/MAKEDEV /lib/dev-state/MAKEDEV
                # This is not needed anymore...
                #keepdir /lib/dev-state/pts /lib/dev-state/shm
                keepdir_mount /dev /dev/pts /dev/shm
        else
                # Normal
                keepdir_mount /dev /dev/pts /dev/shm
                #dosym ../sbin/MAKEDEV /dev/MAKEDEV
        fi
</code>

It seems like both the branches does the same thing (the commented rows indicates that they have been doing different things).
Is that expected? If so, why have the 'if' statement at all?

Then I noticed the 'keepdir_mount ... /dev/pts ...' call.
On a devfs based 2.4 kernel system I suppose that /dev/pts is part of the devfs based dev filesystem. At least I can't find /dev/pts in the output of "gawk '{ print $2 }' /proc/mounts" on my system, and that's why we do 'keepdir /dev/pts' which fails over here.

I don't know how this is supposed to work, but that's something I think you know very well... ;-)
Comment 72 Martin Schlemmer (RETIRED) gentoo-dev 2004-02-27 16:30:25 UTC
Thanks for the research, should be fixed in CVS now.

Can we really start to do something about this right about now?  This is
a portage issue, _NOT_ a baselayout issue.  And one of the biggest reasons
why baselayout has been such an bitch to get to work as a .tbz2.  I really
do not want to go to more extremes to get it to work fine on different
systems (not to talk when it can handle .tbz2, and is used as such).
Comment 73 SpanKY gentoo-dev 2004-02-27 20:35:59 UTC
*** Bug 43093 has been marked as a duplicate of this bug. ***
Comment 74 Jan Schubert 2004-02-28 08:20:25 UTC
What a bummer, it's still not fixed! So here is a quick fix for everyone who is affected (and i guess thats a lot of people): Please edit /usr/portage/sys-apps/baselayout/baselayout-1.8.6.13-r1.ebuild and remove lines 299 to 310 and reemerge baselayout.

HTH,
Jan
Comment 75 Martin Schlemmer (RETIRED) gentoo-dev 2004-02-28 16:27:52 UTC
Andreas, still not fixed your side as well?
Comment 76 Andreas Vinsander 2004-02-29 01:45:30 UTC
Don't know yet! Will try it tomorrow when at work...
Comment 77 Andreas Vinsander 2004-02-29 23:00:03 UTC
Martin, it installed alright here!
Comment 78 oktay altunergil 2004-03-01 09:42:35 UTC
A lof of bugs are listed as a duplicate as this one. It is sort of misleading that the bug that's selected as the master of the duplicates (this one), involves NFS although that does not seem to be the problem.

By the way. Commenting out the mentioned lines solves the problem.

Thanks.

oktay
Comment 79 David Röhr 2004-03-02 01:23:30 UTC
--- /dev/
--- /dev/shm/
!!! copy /var/tmp/portage/baselayout-1.8.6.13-r1/image/dev/shm/.keep -> /dev/shm/.keep failed.
!!! [Errno 13] Permission denied: '/dev/shm/.keep#new'

I also get the error on PPC (2.4.22-ben2), worked fine until this release.
Comment 80 Dead Schorsch 2004-03-10 00:44:22 UTC
Any progress on this? It blocks alsa-driver from installing, so the list of non-usable updates on my system now contains 4 entries: baselayout, alsa-driver, alsa-tools and - apart from this - gramps.
Comment 81 Andrew Sterling Hanenkamp 2004-03-22 15:39:31 UTC
Btw, I recieved a suggestion from one of the Gentoo folks that this bug can be fixed by changing the line:

keepdir /home

to:

keepdir_mount /home

However, this simply will not fix the problem in my environment. The keepdir_mount subroutine checks /proc/mounts to see if "/home" is mounted. In my environment, it is not mounted, but a link to another directory which is mounted. Therefore, the real solution is to also change keepdir_mount() so that it uses readlink. Here's my version which fixed the problem in our environment. This probably needs some cleaning for general use:

keepdir_mount() {
	local x=
	local y=
	local doit=0

	[ -z "$1" ] && return 1

	[ ! -e /proc/mounts ] && return 1

	for y in $*
	do
		doit=0

		if [ $(readlink ${y}) ]
		then
			link=`readlink -f ${y}`
		else
			link=${y}
		fi

		for x in $(gawk '{print $2}' /proc/mounts)
		do
			[ "${x}" = "${link}" ] && doit=1
		done

		if [ "${doit}" -ne 1 ]
		then
			keepdir "${y}"
		fi
	done

	return 0
}
Comment 82 SpanKY gentoo-dev 2004-04-04 15:53:02 UTC
*** Bug 46689 has been marked as a duplicate of this bug. ***
Comment 83 Aron Griffis (RETIRED) gentoo-dev 2004-04-07 13:48:23 UTC
I guess I'm now on the baselayout team with Martin.

Reading through this bug, I'm under the impression that keepdir is broken for mounted read-only (or root_squash) filesystems because portage is unable to add the .keep file to the directory.  keepdir_mount was attempted, but is broken because it assumes that src_install() is running on the same system as pkg_postinst(), and it also doesn't cope with symlinks to mounted filesystems.  Additionally the directories don't show up in the package CONTENTS so they will never be removed automatically.

Since this problem can't really be solved in the ebuild, it's only really solvable in portage.  Here are a couple options:

1. Portage could ignore when directory creation and/or .keep creation fails on a mounted directory.  Anything else should cause merge failure.

2. Should this be tunable via a make.conf variable?  I.e. similar to CONFIG_PROTECT, maybe there should be a variable IGNORE_FAIL which could default to some of the more common directories (/mnt /home /dev etc)
Comment 84 Sascha Silbe 2004-04-07 14:11:52 UTC
How about dropping this .keep stuff and instead only creating the directories during postinstall if they do not already exist?

Comment 85 Aron Griffis (RETIRED) gentoo-dev 2004-04-07 14:52:49 UTC
Regarding comment #84, no, then directories will never be removed by portage because they are not owned by the package according to CONTENTS
Comment 86 Sascha Silbe 2004-04-07 14:57:15 UTC
> then directories will never be removed by portage
I would regard that as a feature, not a bug. :)

On a real system, I would not want Portage to remove any of these directories. It's all right if it creates them (if they don't already exist), but it must not remove them. Even the permissions must remain unchanged.

Comment 87 E. Papegaaij 2004-04-07 15:23:25 UTC
I myself can live with both solutions mentioned in comment #83. On a normal system you would probably not have more than 1 or 2 directories mounted on a read-only fs. So it won't be a problem to put them in make.conf. I don't think it's an option to not remove the directories when unemerging. This would lead to lots and lots of empty directories in for instance /usr/share/doc.
Comment 88 Sascha Silbe 2004-04-07 15:30:26 UTC
> I don't think it's an option to not remove the directories when unemerging. This would lead to lots and lots of empty directories in for instance /usr/share/doc
Since there is no "keepdir" directive for /usr/share/doc/baselayout-<whatever> etc., that's no problem.
We're just talking about initially empty directories that are to be generated during installation. Once they exist, there's no need to change them. Most of them also cannot be removed without damaging the system.

Comment 89 Aron Griffis (RETIRED) gentoo-dev 2004-04-07 15:54:10 UTC
Regarding comment #88, I appreciate the sentiment, but I'm trying to solve this for the general case as well as baselayout.  All you're suggesting is a hack for this specific ebuild, which is probably needed in the interim.  But the portage change is necessary for ebuilds other than baselayout that suffer from the same problems.
Comment 90 E. Papegaaij 2004-04-07 15:56:55 UTC
In reply to comment #88:
There are a lot of keepdir directives in the portage tree. In a worst case scenario you would have all those directories on your system with just a .keep file in it. Some of these directories even depend on the version of the package. This means they will accumulate over time and there is no easy way to remove them. It is true that baselayout creates most of them, but still there are many packages a user might want to uninstall that create directories with 'keepdir', ie. superkaramba, cracklib, tomcat, apache and openoffice.
Comment 91 Jason Stubbs (RETIRED) gentoo-dev 2004-04-07 21:26:55 UTC
Created attachment 28872 [details, diff]
Portage-2.0.50-r4 patch to ignore failed directory or .keep file creation

This patch works by ignoring failures on directory creation or ".keep" files.
Instead it only fails when a regular file copy fails.

Here's sample output from baselayout with /mnt and /etc/skel mounted as
read-only tmpfs:
>>> /bin/rc-status
--- /mnt/
!!! /mnt/cdrom/
!!! /mnt/cdrom/.keep
!!! /mnt/floppy/
!!! /mnt/floppy/.keep
!!! /mnt/.keep
--- /etc/
--- /etc/skel/

!!! Cannot write to '/etc/skel/.bashrc'.
!!! Please check permissions and directories for broken symlinks.
!!! You may start the merge process again by using ebuild:
!!! ebuild /usr/portage/sys-apps/baselayout/baselayout-1.8.6.13-r1.ebuild merge
!!! And finish by running this: env-update

Negative side-effects are that /mnt/{cdrom,floppy} were not created and there
was minimal notification (if /etc/skel is r/w it completes successfully).
Whether this is acceptable is another matter.
Comment 92 Seemant Kulleen (RETIRED) gentoo-dev 2004-04-07 21:57:02 UTC
removing azarah and aron from cc, as they are both in base-system
Comment 93 Jason Stubbs (RETIRED) gentoo-dev 2004-04-07 22:40:48 UTC
Created attachment 28874 [details, diff]
patch to die AFTER package is merged

This patch is similar to the previous one, except that it will die after the
package is merged so that failures on packages such as *cron and postfix are
easily recognised and addressable before using the package. Output is as
follows:

>>> Merging sys-apps/baselayout-1.8.7 to /
<snip>
--- /mnt/
!!! /mnt/cdrom/
!!! /mnt/cdrom/.keep
!!! /mnt/floppy/
!!! /mnt/floppy/.keep
!!! /mnt/.keep
<snip>
>>> Safely unmerging already-installed instance...
<snip>
>>> original instance of package unmerged safely.
<snip>
>>> sys-apps/baselayout-1.8.7 merged.

!!! Creation of the following directories failed.
!!! Please adjust permissions and re-emerge this
!!! this package or create the directories manually.
!!!   /mnt/cdrom
!!!   /mnt/floppy
Comment 94 Christian Pötzsch 2004-04-08 02:40:08 UTC
Same Proplem with /home mounted on nfs fileserver without root access.
When will this bug fixed. IMHO changing keepdir's to keepdir_mount resolv the home  mount problem. In our environment 10-20 users are logged in and there is no possibilty to stop the automounter.

Comment 95 E. Papegaaij 2004-04-08 03:50:28 UTC
After a second thought, I think the best solution will be to add a config variable to mark read-only directories. Failing after emerging is not very practical, because it will stop upgrades of multiple packages.
Comment 96 Sascha Silbe 2004-04-08 04:10:33 UTC
Re comment #89:
For the general case you're right, of course. I was just talking about baselayout.
What other packages do have the same problem (i.e. trying to re-create existing read-only directories)?

What about modifiying keepdir in Portage so only the directory is created (no .keep file) and only if it does not already exist?
That would probably require additional information in binary packages, but seems to be the cleanest solution.


Comment 97 Jason Stubbs (RETIRED) gentoo-dev 2004-04-08 07:40:20 UTC
I should have made it more clear, but my patch will not fail if all directories exist even if the .keep file could not be written.

Specifically, the logic goes:
When creating directories, ignore failures.
When copying .keep files, check if directory exists upon failure.
When copying other files, fail straight away if a file cannot be copied.
If any .keep files couldn't be copied and the containing directories don't exist, fail after the emerge has been completed.

This can be seen by both example outputs. The second one shows that /mnt/.keep could not be created but is not in the list of missing directories.

I think this logic would stop baselayout from failing for every single report in this bug. In the extreme case where a directory does not exist and cannot be created, the user/administrator really needs to know about it.
Comment 98 Marius Mauch (RETIRED) gentoo-dev 2004-04-08 14:32:39 UTC
Just to de-confuse the meaning of keepdir on comment #85:
without the .keep files portage will remove empty dirs on autoclean, this would result in empty dirs being removed on baselayout upgrades for example (it's called KEEPdir for a reason).
As for a good solution, I'd second a IGNORE_FAIL variable (but with no default) in combination with Jason's patch. Not sure if it's necessary to error out or if printing an ewarn and continuing would be better.
Comment 99 Jason Stubbs (RETIRED) gentoo-dev 2004-04-08 19:10:38 UTC
Created attachment 28926 [details, diff]
Same but supports new ebuild var NONCRITICAL_KEEPDIRS

Output is the same as the previous two patches except that the merge will
continue with the next package if the parent of one of the specified dirs is
listed in a new ebuild var NONCRITICAL_KEEPDIRS.

For example, baselayout could specify:
NONCRITICAL_KEEPDIRS="/mnt/floppy /mnt/cdrom" for those specific directories,
NONCRITICAL_KEEPDIRS="/mnt/" for anything UNDER /mnt, or
NONCRITICAL_KEEPDIRS="/mnt" for /mnt and anything under it.
Comment 100 Jason Stubbs (RETIRED) gentoo-dev 2004-04-09 04:51:24 UTC
scratch my explanation of my last patch. it wont work if the var is in the ebuild. s/NONCRITICAL_KEEPDIRS/IGNORE_FAIL/ and it's what Marius described.
Comment 101 Aron Griffis (RETIRED) gentoo-dev 2004-04-09 07:42:14 UTC
*** Bug 47342 has been marked as a duplicate of this bug. ***
Comment 102 Jason Stubbs (RETIRED) gentoo-dev 2004-04-10 01:23:11 UTC
Problems with the patch:
* It doesn't remove -MERGING-baselayout-1.8.6.13-r1
* It doesn't unmerge the previous instance

Dealing with this and still exiting is possible, but would be a bit ugly code-wise.
Are there any objections to a noisy warning and continuing with the emerge?
Comment 103 Jason Stubbs (RETIRED) gentoo-dev 2004-04-11 18:26:52 UTC
Created attachment 29121 [details, diff]
Simpler patch that keeps current behaviour except dying when .keep cannot be written
Comment 104 David Bryson (RETIRED) gentoo-dev 2004-04-12 11:18:48 UTC
To reply to comment 102, by jason

> Problems with the patch:
> * It doesn't remove -MERGING-baselayout-1.8.6.13-r1

that's a file portage leaves hanging around that you need to remove by hand since portage can't seem to deal with finsihing the merge due to the inablity to create .keep

the files live somewhere down in /var/cache or /var/db (I don't recall exactly) and are labeled: 
  -MERGING-<package>
That's how portage knows it hasn't finished merging a package, so if you don't care.. just rm the files.
Comment 105 Aron Griffis (RETIRED) gentoo-dev 2004-04-12 20:54:54 UTC
Btw, to take the pressure off of Jason a bit, baselayout-1.8.9 has a hack that should completely workaround this problem, but for that package only.  Other packages can equally be affected by this problem, so portage isn't completely off the hook ;-)
Comment 106 Aron Griffis (RETIRED) gentoo-dev 2004-04-14 13:57:17 UTC
Now that baselayout has a (working) workaround, reassigning this to the portage team so they can get a real fix in there!
Comment 107 Sascha Silbe 2004-04-14 16:11:16 UTC
baselayout still bails out:

[...]
--- /usr/X11R6/
--- /usr/X11R6/share/
>>> /usr/X11R6/share/info -> ../../share/info
>>> /usr/tmp -> ../var/tmp
>>> /usr/man -> share/man
>>> /usr/doc -> share/doc
>>> /usr/info -> share/info

!!! Cannot write to '/boot'.
!!! Please check permissions and directories for broken symlinks.
!!! You may start the merge process again by using ebuild:
!!! ebuild /usr/portage/sys-apps/baselayout/baselayout-1.8.10.ebuild merge
!!! And finish by running this: env-update


Comment 108 Sascha Silbe 2004-04-15 03:59:31 UTC
I examined the ebuild and found out that /boot is special because baselayout tries to create a SymLink on it:

        # Symlink /boot/boot to . so that /boot can be a separate
        # filesystem and config files still work
        dosym . /boot/boot

/boot is read-only on my system:

/dev/discs/disc0/part2          /boot                   ext2            ro,noatime              1 1


Comment 109 Aron Griffis (RETIRED) gentoo-dev 2004-04-15 07:02:05 UTC
Thansk Sascha, I'll figure out a workaround for that
Comment 110 Aron Griffis (RETIRED) gentoo-dev 2004-04-16 07:47:28 UTC
Sascha, I realized there is no further workaround I can (or am willing, at least) to kludge in the baselayout ebuild.  As it stands it fixes problems at least for directory and .keep file creation.  If I attempt to extend that kludge to symbolic links then the entire ebuild is going to fall apart in my hands.

The only way I think your particular case will be fixed is with IGNORE_FAIL in portage, when that becomes available (hopefully soon!)  Otherwise I'd suggest that rather than mount /boot read-only, you simply refrain from mounting it (similar level of protection, but less choice in the matter than you'd probably like...)  Thanks for your patience.
Comment 111 Sascha Silbe 2004-04-16 08:11:37 UTC
What about just creating that SymLink in postinstall (only) if it does not exist?
I need (read-only) access to /boot, e.g. for System.map.

Comment 112 Jason Stubbs (RETIRED) gentoo-dev 2004-04-16 08:25:26 UTC
I doubt that any portage fix/feature will extend anywhere beyond the keepdir problem. The main question is the best way to implement that.

With the /boot/boot/ symlink, the closest that might occur is that symlinks wont fail if they already exist and point to the right destination.

The main problem is that handling these sorts of abnormalities leads to code that is messy, hard to read and most importantly hard to maintain.
Comment 113 Aron Griffis (RETIRED) gentoo-dev 2004-04-16 14:19:08 UTC
Jason, in comment 87 you said the following:

> When creating directories, ignore failures.
> When copying .keep files, check if directory exists upon failure.
> When copying other files, fail straight away if a file cannot be copied.
> If any .keep files couldn't be copied and the containing directories don't exist, fail after the emerge has been completed.

What if you change that slightly to:

When creating directories, ignore failures.
When copying .keep files, ignore if subpath of IGNORE_FAIL, otherwise forestall failure until after emerge is completed, then fail if parent dir doesn't exist.
When copying files (including symlinks), fail immediately unless it's a subpath of one of the elements of IGNORE_FAIL

Then Sascha could have IGNORE_FAIL=/boot and it would work because portage would ignore ALL failures for that directory.
Comment 114 Sascha Silbe 2004-05-03 13:47:50 UTC
IGNORE_FAIL would be fine for me.

Comment 115 Sascha Silbe 2004-05-05 02:00:22 UTC
The current solution breaks installation from stage3 (/proc is mounted during extraction):

Unpacking /inst/dist/stage3-x86-20040503.tar.bz2... tar: ./proc/.keep: Cannot open: No such file or directory
tar: Error exit delayed from previous errors


:(

Comment 116 Aron Griffis (RETIRED) gentoo-dev 2004-05-05 09:51:47 UTC
Sascha: Huh?  That has nothing to do with this bug, the directory mentioned just happens to coincide with baselayout.  That's just normal behavior for tar and read-only directories.  Please don't muddy the water.
Comment 117 SpanKY gentoo-dev 2004-05-07 11:31:19 UTC
*** Bug 50369 has been marked as a duplicate of this bug. ***
Comment 118 SpanKY gentoo-dev 2004-10-03 03:19:53 UTC
*** Bug 44778 has been marked as a duplicate of this bug. ***
Comment 119 SpanKY gentoo-dev 2004-10-03 03:21:21 UTC
*** Bug 44321 has been marked as a duplicate of this bug. ***
Comment 120 Aron Griffis (RETIRED) gentoo-dev 2005-05-26 08:38:28 UTC
Chris White brought it to my attention that this bug is still open.  Since we
work around the problem in the baselayout ebuild, there's nothing more for
base-system to do.  The ball is now in the portage court to fix the problem.
Comment 121 Tim Redman 2005-08-06 12:32:50 UTC
Is this even a valid bug anymore, since the 1.8.x baselayout isn't even in
portage anymore?  Or does this problem persist with the current versions of
baselayout and portage?
Comment 122 Zac Medico gentoo-dev 2005-08-10 18:55:26 UTC
None of my ~x86 packages currently own /home or /mnt.  Portage will still
produce a "Cannot write" error and exit if it cannot write to a directory, but
that is reasonable, IMO. 
Comment 123 Jason Stubbs (RETIRED) gentoo-dev 2005-08-11 04:07:27 UTC
Baselayout currently has a hack to work around this bug. I'd say best case, we  
can:  
  
1) Get rid of the .keepdir files (which will happen with the CONTENTS rewrite)  
2) Check appropriate access before beginning the move/copy. 
3) If an error still happens during merge, try to revert the work already 
done. 
Comment 124 Brian Harring (RETIRED) gentoo-dev 2005-08-23 01:12:50 UTC
Check appropriate access falls under collision protect code, or moreso, that
feature can be 'tweaked' to make it so.
Comment 125 Jason Stubbs (RETIRED) gentoo-dev 2005-10-07 07:58:29 UTC
This is a combination of bug #23851 + bug #16162 as far as portage goes. 

*** This bug has been marked as a duplicate of 23851 ***