| Summary: | sys-apps/util-linux-2.20-r1: fsck complains about checking file systems mounted ro | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Matthias Maier <tamiko> |
| Component: | [OLD] Core system | Assignee: | Gentoo's Team for Core System packages <base-system> |
| Status: | RESOLVED OBSOLETE | ||
| Severity: | normal | CC: | alexander, andrew, bobbykent32, cyberbat83, eras, erikdenstore+gbugs, eshkrig, goetzger, itumaykin+gentoo, kfm, michael, mschiff, openrc, polynomial-c, tech31842, ulm, wasundwarum |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 415249 | ||
| Attachments: |
emerge --info
/etc/fstab rc-fsck.log rc.log showing /proc/mounts prior to fsck patch to enable fsck of /usr when mounted ro by initramfs |
||
|
Description
Matthias Maier
2012-04-02 21:30:06 UTC
Created attachment 307551 [details]
emerge --info
Created attachment 307553 [details]
/etc/fstab
Hi, (In reply to comment #0) > For modern udev it is necessary to mount /usr (if it is on a separate > partition) prior to system startup. > > sys-kernel/genkernel-3.4.27 does this just fine. > > However, now the fsck service fails on system startup: Actually, We warn you that there was an operational error, but the fsck service doesn't treat this as a failure. > * Checking local filesystems ... > /dev/mapper/ssd-root: clean, 6772/32768 files, 74767/131072 blocks > /dev/mapper/ssd-usr is mounted. e2fsck: Cannot continue, aborting. > > /dev/mapper/ssd-var: clean, 98464/655360 files, 499701/2621440 blocks > /dev/mapper/volume-home: clean, 136144/131072000 files, 288593473/524288000 > blocks > /dev/mapper/ssd-datengrab: clean, 74789/7208960 files, 5000359/28835840 > blocks > /dev/mapper/ssd-opt: clean, 2515/327680 files, 76151/1310720 blocks > /dev/mapper/ssd-usr--portage: clean, 206279/655360 files, 1765162/2621440 > blocks (check after next mount) > /dev/mapper/ssd-usr--portage--packages: clean, 1521/983040 files, > 2177635/3932160 blocks > * Operational error [ !! ] This output shows that all of your file systems, except /usr, were checked. We let you know that there is an operational error (which there is, from fsck's pov anyway, because we tried to run fsck on /usr which is already mounted), but fsck -a checks all other file systems and the fsck service script exits successfully in this situation and does not treat this as a critical error. I would like to keep the warning, because I dont know all other situations that could cause fsck to exit with exit code 8. How would you suggest keeping the message but not making it appear as a failure message? Ideally, I would tel fsck -a to skip /usr isf it is mounted rw and not report this as a failure. (In reply to comment #3) > Actually, We warn you that there was an operational error, but the fsck > service doesn't treat this as a failure. > > This output shows that all of your file systems, except /usr, were checked. > We let you know that there is an operational error (which there is, from > fsck's pov anyway, because we tried to run fsck on /usr which is already > mounted), but fsck -a checks all other file systems and the fsck service > script exits successfully in this situation and does not treat this as a > critical error. Jup. This is the way I understood the "operational error". I'm sorry to have confused you with "fsck fails" in my bug description. > I would like to keep the warning, because I dont know all other situations > that could cause fsck to exit with exit code 8. > How would you suggest keeping the message but not making it appear as a > failure message? I would leave the error message as it is. The point is, that the genkernel init script mounts /usr ro - at least it is telling this (I haven't checked manually.) So ideally, I would like to have fsck -a just succeed in checking /usr as it succeeds in checking the already (ro) mounted /. Does fsck -a have a problem with any other partition (beside /) already mounted? rw obviously, but ro? > > Ideally, I would tel fsck -a to skip /usr isf it is mounted rw and not > report this as a failure. I wouldn't do this. It is always possible to skip a rw mounted partition / or any other special partition with a passno of 0 in fstab. (In reply to comment #4) > The point is, that the genkernel init script mounts /usr ro - at least it is > telling this (I haven't checked manually.) > > So ideally, I would like to have fsck -a just succeed in checking /usr as it > succeeds in checking the already (ro) mounted /. > > Does fsck -a have a problem with any other partition (beside /) already > mounted? rw obviously, but ro? Apparently it does. I checked my rc.log and have the same issue. I also attempted to manually fsck a partition that was mounted ro and fsck complained very loudly about how if I do this I will cause serious file system damage. This is not a bug in OpenRc but an issue with fsck. I will send this bug to base-system. All, it appears that fsck complains loudly when we ask it to check a read-only filesystem. This causes confusing results when fsck is run in the latest openrc because we report an operational error but the fsck service does not fail. I will attach the last log of my boot runlevel showing this. Also,it is possible to reproduce the complaint by mounting a filesystem ro then trying to run fsck on that file system. If you do this you will get a big warning about how you will cause file system damage if you continue. Created attachment 308663 [details]
rc-fsck.log
This is the fsck portion of my rc.log the last time I booted my system.
Note that fsck complains about /dev/sda6. This is my /usr file system
which is mounted ro by genkernel in the initramfs.
and you're sure it's mounted ro ? compare /proc/mounts and /etc/mtab and see what you get for that mount point. @vapier: On my system, /etc/mtab is a file, and not a symbolic link. If I mount something ro, it shows up as ro in /proc/mounts but not in /etc/mtab . The mtab init script in openrc allows /etc/mtab to existas a symbolic link, but if it exists as a file, it gets updated with a snapshot of /proc/mounts, without tmpfs entries, as part of the boot process. The mtab script runs *after* fsck and after all local file systems are mounted, so, it looks like we do have a bug in openrc. If mtab is a file, it will show /usr mounted rw while /proc/mounts shows it mounted ro. I see two ways out of this: I can require everyone to link /etc/mtab->/proc/mounts, or I can filter the /usr file system out of /etc/mtab. What are your thoughts? (In reply to comment #9) you cannot require /etc/mtab be a symlink (yet). there's still information in there that userland mount stores that is not mentioned in /proc/mounts. before the boot runlevel finishes processing, you can do whatever you like to /etc/mtab. but once everything has finished booting, all real mounts must be reflected in /etc/mtab (including /usr). (In reply to comment #10) > before the boot runlevel finishes processing, you can do whatever you like > to /etc/mtab. but once everything has finished booting, all real mounts > must be reflected in /etc/mtab (including /usr). The issue is that /etc/mtab needs to be manipulated before fsck runs to make sure that the settings for /usr match those in /proc/mounts. Root is still ro at that point, so the only way I could fis this would be to remount root rw, do the fix, then remount root ro. That seems not to be a good idea to me. What are your thoughts? (In reply to comment #11) > The issue is that /etc/mtab needs to be manipulated before fsck runs to make > sure that the settings for /usr match those in /proc/mounts. Is this really the issue? On my system /etc/mtab is already a symbolic link pointing to /proc/self/mounts. > Root is still ro at that point, so the only way I could fis this would be to > remount root rw, do the fix, then remount root ro. (In reply to comment #8) > and you're sure it's mounted ro ? compare /proc/mounts and /etc/mtab and > see what you get for that mount point. I've just tested this. /usr is indeed mounted ro at the time fsck starts. See the next attachment. Created attachment 308745 [details]
rc.log showing /proc/mounts prior to fsck
rc.log with a hook showing /proc/mounts and /etc/mtab (which is a symlink to /proc/self/mounts, hence the same output as /proc/mounts) prior to fsck
(In reply to comment #13) > (In reply to comment #8) > > and you're sure it's mounted ro ? compare /proc/mounts and /etc/mtab and > > see what you get for that mount point. > > I've just tested this. /usr is indeed mounted ro at the time fsck starts. > See the next attachment. In that case, sorry about all of the extra noise on this bug. This is definitely an fsck issue; there isn't much I can do in OpenRC. fsck runs, -a runs, but it is reporting errors when checking ro mounted file systems. If fsck can handle ro-/ a patch should be trivial to also let it accept ro-/usr the same way. I consider this a severe issue for all sperate-/usr users because /usr will never be checked even after a crash anymore a users will work with broken /usr filesystems which is a very bad thing. Lets hope a patch will be available soonish. the trouble is that patching `fsck` won't help i don't think. that's just a multiplexing program that turns around and runs `fsck.<fs>`, and that fs-specific fsck is what does the mount check. i'll have to query on the util-linux mailing list to see if they have any ideas. (In reply to comment #17) Hi, I don't quite understand why you changed the bug summary to " fsck complains about checking file systems mounted ro when /etc/mtab says rw" The behaviour I observed was more that at system startup fsck complains about any filesystem mounted ro to any other location than /. I don't think that this has to do anything with /etc/mtab at all. The reasons: 1.) In my case /etc/mtab is a symlink to /proc/self/mounts - so it should not contain an invalid entry. 2.) The root file system is remounted rw _after_ fsck. Before that there is no way to update /etc/mtab. And /etc/mtab must be assumed to be invalid (e.g. a hard crash). fsck usually handles this cases quite fine. vapier: I was considering the problem, and having some better way of telling which subset of mountpoints fsck -a should run on, would greatly improve matters. For a given boot, being able to tell it "I've already checked A,B,C" would help cover it, as well as better control over RO-check behavior. (In reply to comment #17) > the trouble is that patching `fsck` won't help i don't think. that's just a > multiplexing program that turns around and runs `fsck.<fs>`, and that > fs-specific fsck is what does the mount check. > > i'll have to query on the util-linux mailing list to see if they have any > ideas. Do you have any updates on this? It seems like fsck* could be modified to check any read-only file system like it already checks /. *** Bug 443018 has been marked as a duplicate of this bug. *** Seeing this as well. I have an initramfs built by genkernel (v 3.4.45) and /usr is listed in /etc/initramfs.mounts (along with /var and /opt - there are fsck complaints about these as well as /usr). Is there any news on a fix? Did some digging and noticed the following in e2fsck/unix.c from e2fsck-1.42 (which is installed on my system):
if ((!(ctx->mount_flags & EXT2_MF_MOUNTED)) ||
((ctx->mount_flags & EXT2_MF_ISROOT) &&
(ctx->mount_flags & EXT2_MF_READONLY) &&
!(ctx->options & E2F_OPT_WRITECHECK)))
return;
if ((ctx->options & E2F_OPT_READONLY) &&
!(ctx->options & E2F_OPT_WRITECHECK)) {
printf(_("Warning! %s is mounted.\n"), ctx->filesystem_name);
return;
}
Note that for / ctx->mount_flags is checked for read only, whereas for other filesystems ctx->options is used.
I modified the code to:
if ((!(ctx->mount_flags & EXT2_MF_MOUNTED)) ||
((ctx->mount_flags & EXT2_MF_READONLY) &&
!(ctx->options & E2F_OPT_WRITECHECK))) {
if (!(ctx->mount_flags & EXT2_MF_ISROOT))
printf(_("Warning! %s is mounted.\n"), ctx->filesystem_name);
return;
}
and did not get a "WARNING!!! The filesystem is mounted. If you continue you ***WILL*** cause ***SEVERE*** filesystem damage." message, instead it returned the somewhat less alarming warning in the printf statement.
When the filesystem in question is mounted with "ro" option I see:
ctx->mount_flags & EXT2_MF_READONLY is TRUE.
ctx->options & E2F_OPT_READONLY is FALSE.
when mounted regularly (i.e. "rw") I see:
ctx->mount_flags & EXT2_MF_READONLY is FALSE.
ctx->options & E2F_OPT_READONLY is FALSE.
I've not tried patching e2fsck yet (in my /sbin, the files e2fsck, fsck.ext2 fsck.ext3, fsck.ext4 and fsck.ext4dev are identical), though I'm wondering if this might have something to do with the complaint about /usr on boot.
More digging, looks like e2fsprogs-1.42.6 might do something similar:
if ((!(ctx->mount_flags & (EXT2_MF_MOUNTED | EXT2_MF_BUSY))) ||
((ctx->mount_flags & EXT2_MF_ISROOT) &&
(ctx->mount_flags & EXT2_MF_READONLY) &&
!(ctx->options & E2F_OPT_WRITECHECK)))
return;
if (((ctx->options & E2F_OPT_READONLY) ||
((ctx->options & E2F_OPT_FORCE) &&
(ctx->mount_flags & EXT2_MF_READONLY))) &&
!(ctx->options & E2F_OPT_WRITECHECK)) {
log_out(ctx, _("Warning! %s is %s.\n"),
ctx->filesystem_name,
ctx->mount_flags & EXT2_MF_MOUNTED ?
"mounted" : "in use");
return;
}
though it looks like it requires a force option somewhere ...
I'll need to reboot to see whether fsck is forced in the genkernel initramfs: # date ; mount -o "ro,noauto,noatime" /dev/sdg4 /next ; date ; fsck /dev/sdg4 ; date ; fsck -nf /dev/sdg4 ; date ; umount /next Wed Jan 16 19:08:49 EST 2013 Wed Jan 16 19:08:49 EST 2013 fsck from util-linux 2.21.2 e2fsck 1.42.6 (21-Sep-2012) /dev/sdg4 is mounted. e2fsck: Cannot continue, aborting. Wed Jan 16 19:08:49 EST 2013 fsck from util-linux 2.21.2 e2fsck 1.42.6 (21-Sep-2012) Warning! /dev/sdg4 is mounted. Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information /dev/sdg4: 6270/1155072 files (0.0% non-contiguous), 166536/4612663 blocks Wed Jan 16 19:08:51 EST 2013 Have to wonder why it's acceptable to fsck the root filesystem when it's mounted ro and not other filesystems when they are mounted ro. Created attachment 336242 [details, diff] patch to enable fsck of /usr when mounted ro by initramfs Patch attached for e2fsprogs-1.42.6. After applying this, fsck runs with a warning on /usr during boot. No guarantees that it does something else to fsck and allows it to run in situations where it shouldn't. Also started a thread on the e2fsprogs sourceforge Help forum (https://sourceforge.net/p/e2fsprogs/discussion/7053/thread/138e160a/) to discuss the patch. Hello, any solutions yet? I must check the system manually Theodore Ts'o's comment at the disussion started by BobbyK is quite insightful on why enabling fsck for read-only mounted file systems is not a good solution [1]. At the moment I only see one proper fix for that: - Let the initramfs check every file system before it is mounted - Later in openrc, silently skip all already mounted file systems in fsck. See bug 415249 for a discussion on that. *sigh* (I'll learn it someday) [1] http://sourceforge.net/p/e2fsprogs/discussion/7053/thread/138e160a/#35ed Any progress to solve the Problem? Hello, is there any Solutions in Portage available? Regards (In reply to Matthias Maier from comment #28) > Theodore Ts'o's comment at the disussion started by BobbyK is quite > insightful on why enabling fsck for read-only mounted file systems is not > a good solution [1]. Sorry, I don't find his comment particularly insightful. He raises two valid points, which however are self-evident: - Trying to repair an rw mounted fs is not a good idea. - After an ro mounted fs has been repaired, the system must be rebooted. The rest of his argumentation isn't compelling at all: If the init system is buggy and fails to reboot the system (in the second case above), then of course there will be problems. But I fail to see why fscking ro-mounted root and separate /usr would result in "a rickety system set up using mucilage and bailing wire" whereas fscking a large root (including /usr) wouldn't. Let's close this bug report. |
