Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 581904 - sys-boot/grub: grub2-mkconfig generates invalid grub.conf when using btrfs multidevice
Summary: sys-boot/grub: grub2-mkconfig generates invalid grub.conf when using btrfs mu...
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 1 vote (vote)
Assignee: Mike Gilbert
URL: https://savannah.gnu.org/bugs/?39591
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-05-03 03:29 UTC by Michael Jones
Modified: 2017-08-25 18:36 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Jones 2016-05-03 03:29:12 UTC
grub2-mkconfig generates a grub.conf file that can't be used to boot when used on a root filesystem that spans multiple harddrives, like in a btrfs-raid1 configuration.



        menuentry 'Gentoo GNU/Linux, with Linux 4.4.8-hardened-r1-0' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-4.4.8-hardened-r1-0-advanced-0c1b6123-85a9-4dcc-9f82-4328cd3f8365' {
                load_video
                insmod gzio
                insmod part_gpt
                insmod part_gpt
                insmod part_gpt
                insmod part_gpt
                insmod part_gpt
                insmod part_gpt
                insmod btrfs
                set root='hd4,gpt3'
                if [ x$feature_platform_search_hint = xy ]; then
                  search --no-floppy --fs-uuid --set=root --hint-bios=hd4,gpt3 --hint-efi=hd4,gpt3 --hint-baremetal=ahci4,gpt3  --hint-bios=hd5,gpt3 --hint-efi=hd5,gpt3 --hint-baremetal=ahci5,gpt3  --hint-bios=hd2,gpt3 --hint-efi=hd2,gpt3$
                else
                  search --no-floppy --fs-uuid --set=root 0c1b6123-85a9-4dcc-9f82-4328cd3f8365
                fi
                echo    'Loading Linux 4.4.8-hardened-r1-0 ...'
                linux   /@/boot/kernel-4.4.8-hardened-r1-0 root=/dev/sde3
        /dev/sdf3
        /dev/sdc3
        /dev/sdd3
        /dev/sdb3
        /dev/sda3 ro rootflags=subvol=@ init=/usr/lib/systemd/systemd zcache zswap.enabled=1 rootfstype=btrfs root=UUID="0c1b6123-85a9-4dcc-9f82-4328cd3f8365" rw
        }
Comment 1 Mike Gilbert gentoo-dev 2016-05-03 12:32:14 UTC
Make sure you have an initramfs. Grub-mkconfig will then generate a root=UUID=... entry. 

You will need an initramfs to properly initialize a multi device btrfs system anyway.
Comment 2 Michael Jones 2016-05-03 17:36:57 UTC
Have one. Used dracut to generate it.

It doesn't generate a root=UUID... entry. I added that manually.

The config grub generates is invalid, but if I remove the multiple "insmod part_gpt" entries, and the extra /dev/sd** entries, it works fine.
Comment 3 Mike Gilbert gentoo-dev 2016-05-03 17:57:16 UTC
(In reply to Michael Jones from comment #2)

Your kernel in /boot has a strange name, so grub-mkconfig is not able to properly relate it to the initramfs file generated by dracut.
Comment 4 S. Mannem 2016-06-21 06:48:37 UTC
I can relate to this issue. I have traced the issue to 
this if statement in /etc/grub.d/10_linux (line 46):
if [ "x${GRUB_DEVICE_UUID}" = "x" ] || [ "x${GRUB_DISABLE_LINUX_UUID}" = "xtrue" ] \
    || ! test -e "/dev/disk/by-uuid/${GRUB_DEVICE_UUID}"
    uses_abstraction "${GRUB_DEVICE}" lvm; then
  LINUX_ROOT_DEVICE=${GRUB_DEVICE}
else
  LINUX_ROOT_DEVICE=UUID=${GRUB_DEVICE_UUID}
fi

If you have btrfs on top of LVM, the last test in the if kicks in 
and LINUX_ROOT_DEVICE=${GRUB_DEVICE}.

I have worked around this by adding this to /etc/default/grub:
GRUB_DEVICE="UUID=5ffbb8a6-c9fe-556f-ab81-85454d04c218"
overriding the default 'find my UUID functionality' from grub2.

I'd like to propose the following:
1: Add the workaround from above to the documentation. Something like:
If you use btrfs with multiple devices for your root disk on top of LVM, you should add the UUID to /etc/default/grub like so:
GRUB_DEVICE="UUID=5ddbb8a6-c9fe-447f-ab81-85454d04c218"

or

2: Change /etc/grub.d/10_linux:
diff /etc/grub.d/10_linux etc_grub.d_10_linux
47,48c47,51
<     || ! test -e "/dev/disk/by-uuid/${GRUB_DEVICE_UUID}"
<     uses_abstraction "${GRUB_DEVICE}" lvm; then
---
>     || ! test -e "/dev/disk/by-uuid/${GRUB_DEVICE_UUID}"; then
>   LINUX_ROOT_DEVICE=${GRUB_DEVICE}
> elif [ x"$GRUB_FS" = 'xbtrfs' ]; then
>   LINUX_ROOT_DEVICE=UUID=${GRUB_DEVICE_UUID}
> elif uses_abstraction "${GRUB_DEVICE}" lvm; then

Making the if statement like:
if [ "x${GRUB_DEVICE_UUID}" = "x" ] || [ "x${GRUB_DISABLE_LINUX_UUID}" = "xtrue" ] \
    || ! test -e "/dev/disk/by-uuid/${GRUB_DEVICE_UUID}"; then
  LINUX_ROOT_DEVICE=${GRUB_DEVICE}
elif [ x"$GRUB_FS" = 'xbtrfs' ]; then
  LINUX_ROOT_DEVICE=UUID=${GRUB_DEVICE_UUID}
elif uses_abstraction "${GRUB_DEVICE}" lvm; then
  LINUX_ROOT_DEVICE=${GRUB_DEVICE}
else
  LINUX_ROOT_DEVICE=UUID=${GRUB_DEVICE_UUID}
fi

Since BTRFS works better with UUID's, use this (even when on top of LVM.

or

3: Leave out 'uses_abstraction "${GRUB_DEVICE}" lvm' altogether.
I don't know why you should not use UUID= with LVM.

Please let me know if I should clear out anymore, or how to proceed with this?
Comment 5 Hannes Schweizer 2017-06-11 18:40:31 UTC
Confirmed here as well, but it's definitely an upstream issue.

The only way to fix this, without manually forcing GRUB_DEVICE in /etc/default/grub (which is not an option for me, because it's synced to multiple machines), is to modify grub-mkconfig as already proposed in the initial upstream bug-report in 2013 (https://savannah.gnu.org/bugs/?39591).

Seems like the cleanest solution in my POV.

https://github.com/gentoo-mirror/gentoo/pull/4

Enough grub for one day...
Comment 6 Hannes Schweizer 2017-06-11 23:58:33 UTC
pushed to wrong repo, here's the right pull request:
https://github.com/gentoo/gentoo/pull/4913
Comment 7 Mike Gilbert gentoo-dev 2017-06-18 03:18:06 UTC
(In reply to Hannes Schweizer from comment #6)
> pushed to wrong repo, here's the right pull request:
> https://github.com/gentoo/gentoo/pull/4913

I left some feedback on the PR several days ago.

Also, please submit the patch upstream on the grub-devel mailing list.

https://lists.gnu.org/mailman/listinfo/grub-devel
Comment 8 Hannes Schweizer 2017-07-10 21:33:17 UTC
Sry for the delayed response...
I've created a new pull request (#5073), including a new ebuild revision and a proper patch against the upstream git.

I've submitted the patch to the upstream mailing list as well:
https://lists.gnu.org/archive/html/grub-devel/2017-07/msg00010.html

Thanks for your guidance!
Comment 9 Mike Gilbert gentoo-dev 2017-08-25 18:36:15 UTC
So thinking on this some more, I'm not sure it is safe to just take the first device node returned.

Specifically for the root kernel parameter, telling the kernel about a single device in a multi-device btrfs is a bit dangerous. The btrfs wiki mentions that all devices must be passed via the options parameter (rootflags) if an initramfs is not used to initiate a scan.

https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices