Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 99682 - Halt.sh patch allows proper unionfs support on LiveCD
Summary: Halt.sh patch allows proper unionfs support on LiveCD
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] baselayout (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
: 119799 (view as bug list)
Depends on:
Blocks:
 
Reported: 2005-07-20 07:55 UTC by Chris Gianelloni (RETIRED)
Modified: 2006-08-17 08:54 UTC (History)
13 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
halt.patch (halt.patch,802 bytes, patch)
2005-07-20 07:56 UTC, Chris Gianelloni (RETIRED)
Details | Diff
a patch agaist 1.11.13 (halt.sh.diff,1.22 KB, patch)
2005-07-28 11:34 UTC, Rafael Ávila de Espíndola
Details | Diff
halt_sh-unionfs.patch (halt_sh-unionfs.patch,2.23 KB, patch)
2005-08-01 09:34 UTC, Martin Schlemmer (RETIRED)
Details | Diff
halt_sh-unionfs-2.patch (halt_sh-unionfs.patch,2.49 KB, patch)
2005-08-01 11:10 UTC, Martin Schlemmer (RETIRED)
Details | Diff
halt_sh-unionfs-3.patch (halt_sh-unionfs.patch,2.28 KB, patch)
2005-08-01 13:56 UTC, Martin Schlemmer (RETIRED)
Details | Diff
halt_sh-unionfs-4.patch (halt_sh-unionfs.patch,2.76 KB, patch)
2005-08-01 14:15 UTC, Martin Schlemmer (RETIRED)
Details | Diff
use uninoctl to remove /memory (halt_sh-unionfs-5.patch,2.84 KB, patch)
2005-08-02 07:21 UTC, Rafael Ávila de Espíndola
Details | Diff
halt.sh-5 patch (halt.sh.patch,3.59 KB, patch)
2006-02-09 16:06 UTC, Jean-François Richard
Details | Diff
netmount patch (netmount.patch,1.91 KB, patch)
2006-02-09 16:07 UTC, Jean-François Richard
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Chris Gianelloni (RETIRED) gentoo-dev 2005-07-20 07:55:20 UTC
The summary pretty much says it all.  The following patch allows for proper
shutdown and unmounting on a unionfs-enabled LiveCD.  Blame rocket.
Comment 1 Chris Gianelloni (RETIRED) gentoo-dev 2005-07-20 07:56:57 UTC
Created attachment 63894 [details, diff]
halt.patch

By the way, this was against sys-apps/baselayout-1.11.12-r4

We would *love* to have a newer revision without needing to upgrade to the new
baselayout, simply because of the changes in all of the wireless tools, but
aren't requiring it.  We just need to know if we need to update something in
addition to baselayout in our release snapshot.

Thanks
Comment 2 Rafael Ávila de Espíndola 2005-07-28 11:34:21 UTC
Created attachment 64565 [details, diff]
a patch agaist 1.11.13

This patch also has an additional check to prevent fuser from killing every
process or failing if /memory or /newroot doesn't exist.
Comment 3 Martin Schlemmer (RETIRED) gentoo-dev 2005-08-01 09:34:19 UTC
Created attachment 64879 [details, diff]
halt_sh-unionfs.patch

I do not like it.  You guys just keep on piling 'special' directories :/  PS,
you guys sure /memory is what you want? And why?

Anyhow, how about attached patch .. not tested btw.
Comment 4 Chris Gianelloni (RETIRED) gentoo-dev 2005-08-01 10:09:09 UTC
Honestly, rocket had written the original patch, I just filed the bug.
Comment 5 Martin Schlemmer (RETIRED) gentoo-dev 2005-08-01 11:10:32 UTC
Created attachment 64883 [details, diff]
halt_sh-unionfs-2.patch

Small tweaks.
Comment 6 Martin Schlemmer (RETIRED) gentoo-dev 2005-08-01 13:56:38 UTC
Created attachment 64895 [details, diff]
halt_sh-unionfs-3.patch

Note that you _NEED_ to set RC_NO_UMOUNTS in livecd-functions.sh (but only when
you are actually booting off the CD - [[ -n ${CDBOOT} ]] - else the default
should be fine).
Comment 7 Martin Schlemmer (RETIRED) gentoo-dev 2005-08-01 14:15:30 UTC
Created attachment 64897 [details, diff]
halt_sh-unionfs-4.patch

Some more cleanups.
Comment 8 Rafael Ávila de Espíndola 2005-08-02 07:21:55 UTC
Created attachment 64934 [details, diff]
use uninoctl to remove /memory

Small modifications to the previous patch:
1) add rootfs to RC_NO_UMOUNT_FS. If the / is a unionfs this prevents the
script from trying to remount / ro.
2) run unionctl /usr --remove 0
This a hack permit the unmounting of /memory. Something like
num = $(unionctl / --list | cat -n | grep "(rw)$" | cut -f 1)
num = $((num - 1))
might be more reliable
Comment 9 Martin Schlemmer (RETIRED) gentoo-dev 2005-08-03 12:27:06 UTC
You cannot just run unionctl explicitly .. some people may not have it. 
Something like:

if [[ -x <path_to>/unionctl && $(gawk '$3 == "sysfs" { print "yes"; exit }'
/proc/mounts) == "yes" ]] ; then
        unionctl /usr --remove 0
fi

?
Comment 10 Martin Schlemmer (RETIRED) gentoo-dev 2005-08-31 08:46:16 UTC
Any update on this ?
Comment 11 Chris Gianelloni (RETIRED) gentoo-dev 2005-08-31 09:45:24 UTC
No clue here... I just submitted the thing... ;]
Comment 12 Rafael Ávila de Espíndola 2005-08-31 18:23:10 UTC
I have a small system running with Martin Schlemmer's patch plus the  
additional hack that I have mentioned.  
 
Comment 13 Chris Gianelloni (RETIRED) gentoo-dev 2005-12-21 07:56:27 UTC
OK...

So I've been told that this is waiting on release engineering for something... what are we waiting on?
Comment 14 SpanKY gentoo-dev 2005-12-21 07:58:37 UTC
testing

you have to say whether the posted patches work for you
Comment 15 SpanKY gentoo-dev 2006-01-23 10:46:22 UTC
*** Bug 119799 has been marked as a duplicate of this bug. ***
Comment 16 Jean-François Richard 2006-02-03 13:11:58 UTC
I'm not quite sure where to put this bug report (should I open a new one?).

The thing is that I'm using the unionfs mechanism to overlay a rw filesystem over an NFS ro root (instead of a LiveCD root).  The rw overlay filesystem is also mounted from NFS, all by the initramfs.  See bug # 121334.

The above patches seem to fix the halt.sh problem, but to have a completely "all green" rebooting, the "nfsmount" script also needs to take those RC_NO_UMOUNTS directories into account.

My patch would be, on "/etc/init.d/nfsmount":

stop() {
        local err=0
        local unmounts
        ebegin "Unmounting NFS filesystems"

        # taken from halt.sh
        RC_NO_UMOUNTS=${RC_NO_UMOUNTS:-/newroot|/memory|/}
        unmounts=$(awk -v NO_UMOUNTS="${RC_NO_UMOUNTS}" \
                '{ \
                    if (($2 !~ NO_UMOUNTS) && \
                        ($3 == "nfs")) \
                      print $2 \
                }' /proc/mounts | sort -ur)
        for x in ${unmounts}; do
                if ! /sbin/umount "${x}" &>/dev/null; then
                        err=1
                fi
        done 

        eend ${err} "Error unmounting NFS filesystems"
        stop_statd
}
Comment 17 Jean-François Richard 2006-02-03 14:12:01 UTC
Well, in fact there is also a "netmount" that I just discovered!

(Is nfsmount a deprecated script?)

The modification would be something like this (note that I use the exact same variables as in halt.sh, perhaps these should go global somewhere?)  Also I think RC_NO_UMOUNT_FS is necessary, because searching for "nfs" also yields "unionfs".  With this $3 !~ RC_NO_UMOUNT_FS, it should not happen.

while [[ -n ${remaining} && ${retry} -lt 3 ]] ; do
    # Populate $remaining with a newline-delimited list of network
    # filesystems.  Mount points have spaces swapped for '\040' (see
    # fstab(5)) so we have to translate them back to spaces.
    RC_NO_UMOUNTS=${RC_NO_UMOUNTS:-/newroot|/memory|/}
    RC_NO_UMOUNT_FS="^(proc|devpts|sysfs|devfs|tmpfs|usb(dev)?fs|unionfs)$"
    remaining="$(awk -v NO_UMOUNTS='^(${RC_NO_UMOUNTS})$' \
        '$3 ~ /'${NET_FS_LIST// /|}'/   \
            { \
                if (($3 !~ RC_NO_UMOUNT_FS) && \
                    ($2 !~ NO_UMOUNTS)) \
                   print $2 \
            }' /proc/mounts | sort -r)"
Comment 18 Jean-François Richard 2006-02-03 14:17:59 UTC
Also, with this last modification on netmount, I still get the "Failed to simply unmount filesystems" error, even though it is normal in this case... (you just can't unmount "/" or the initramfs "/memory")  Would it be a good idea to have it be informational instead of a big bold red error that makes grandma fear her PC is broken?
Comment 19 Jean-François Richard 2006-02-09 16:04:55 UTC
Please forget my comment #17, there was an awk flaw in it.


netmount patch
--------------

The new patch proposed will not try to unmount every network fs if you set
RC_SIMPLE_NET_UMOUNT="no" in /etc/conf.d/rc.  It will skip the 'umount -art', and go directly in the next mode, where there is a fine-grained control on filesystems being unmounted.  It then uses a similar technique as the one proposed for inclusion in halt.sh.

It will not try to unmount any network fs that is under /.unions/, since all the "/" overlays will probably be stored there (if bug # 121334 gets accepted -- for now /.unions/ contains only what was known as "/memory", but perhaps somebody will need multiple root overlays one day (me!)).  The halt.sh script will take care of "unmounting" the unions.


halt.sh patch
-------------

This patch contains the previous patches submitted in this bug report, plus a little section that does remount every branches of every unionfs overlays as read-only.

I tried to _remove_ the branches instead of making them RO, but it caused panics and oops at reboot.  Unionfs is probably not happy with the root overlay being moved... inside that same root (ref. bug # 121334).  



I would appreciate to have feedback and suggestions, this is all experimental work I do to have an elegant Gentoo diskless clustering solution and I would like to make sure that it won't break other things, such as LiveCD booting.

As for now, I am able to get a nice boot and reboot with a RW 'state-saving' unionfs mounted from NFS, overlayed over a RO 'base system' /, also mounted from NFS.

I guess one could do the same NFS RW over RO root from a LiveCD, which is a nice way to incrementally customize the CD by having its state saved somewhere.  Perhaps this method could deprecate the use of "add" tarballs pioneered with the NFS-mounted LiveCD.

Comment 20 Jean-François Richard 2006-02-09 16:06:14 UTC
Created attachment 79381 [details, diff]
halt.sh-5 patch
Comment 21 Jean-François Richard 2006-02-09 16:07:45 UTC
Created attachment 79382 [details, diff]
netmount patch
Comment 22 Michael Hordijk 2006-02-24 06:24:58 UTC
I'm in the same boat, as Jean-Fran
Comment 23 Michael Hordijk 2006-02-24 06:24:58 UTC
I'm in the same boat, as Jean-François Richard, using gentoo with root over nfs and a unionfs overlay.  These scripts work well and seem to be a pretty decent solution to the same problems I was having.  I suppose this is a "works for me" vote.
Comment 24 Luca 2006-05-16 16:19:29 UTC
Hi to all!
I have use your patch for my live distro and them works very fine: right now i can reboot or shout down the pc with no problem.
The only thing it's that i got those error during the reboot process:

Umounting loopback filesystem
Failed to umount 

It tries to umount loopback other 3 times with no resoult, then it moves on..The system reboot fine, but it slow the process of about 1 min with all those retry.

Any idea?
Comment 25 Chris Gianelloni (RETIRED) gentoo-dev 2006-05-19 06:25:54 UTC
What version of genkernel are you guys using with this patch and *exactly* which patch(es) are you guys using?
Comment 26 Luca 2006-05-20 03:35:17 UTC
I used those patch:
halt.sh-5 patch
netmount patch

With kernel 2.6.16 on genkernel 3.3.11d

the system tries to umount loopback filesystem till moves on..
Comment 27 Roy Marples (RETIRED) gentoo-dev 2006-05-23 01:57:50 UTC
(In reply to comment #25)
> the system tries to umount loopback filesystem till moves on..

Could you also try the patch here?
http://bugs.gentoo.org/attachment.cgi?id=77698&action=view

You may have to manually apply.

Comment 28 Roy Marples (RETIRED) gentoo-dev 2006-05-23 06:29:31 UTC
baselayout-1.12.0 has had a slightly modified halt_sh-unionfs-4.patch applied. We also stop the unionfs addon if present - which means that if unionfs installs /lib/rcscripts/addons/unionfs-stop.sh then it will remount all existing unionfs branches as read-only.

The netmount script has not been patched at this time.
Comment 29 Roy Marples (RETIRED) gentoo-dev 2006-08-17 08:54:29 UTC
baselayout-1.12.4 has this in now.