Created attachment 849103 [details] this is the grub.cfg file created When running grub-mkconfig, any version of os-prober available in portage will detect the operating system in the subvolume but it will not actually make an entry for it or will not make it properly. # grub-mkconfig -o /boot/grub/grub.cfg Generating grub configuration file ... Found linux image: /boot/vmlinuz-5.15.88-gentoo-dist Found initrd image: /boot/initramfs-5.15.88-gentoo-dist.img Warning: os-prober will be executed to detect other bootable partitions. Its output will be used to detect bootable binaries on them and create new boot entries. Found Windows Boot Manager on /dev/sda1@/EFI/Microsoft/Boot/bootmgfw.efi Found Arch Linux on /dev/sda5 Found Fedora Linux 37 (Workstation Edition) on /dev/sda5 Adding boot menu entry for UEFI Firmware Settings ... done # ls /mnt/sda5/ '@arch'/ '@dotfiles'/ '@fedora'/ '@gentoo'/ '@swap'/ This is one of the entries it created for ArchLinux. As you can see there is many things wrong about it. Starting from /boot/vmlinuz-linux... it should be @arch/boot/vmlinuz.... Another problem is that it is adding a ^ initrd /boot/intel-ucode.img^/@arch/boot/initramfs-linux-zen.img menuentry 'Arch Linux (on /dev/sda5)' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-simple-abadf235-d80f-4d5e-b85e-5cd82c828d92' { insmod part_gpt insmod btrfs set root='hd0,gpt5' if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt5 --hint-efi=hd0,gpt5 --hint-baremetal=ahci0,gpt5 abadf235-d80f-4d5e-b85e-5cd82c828d92 else search --no-floppy --fs-uuid --set=root abadf235-d80f-4d5e-b85e-5cd82c828d92 fi linux /boot/vmlinuz-linux-zen root=UUID=abadf235-d80f-4d5e-b85e-5cd82c828d92 rw rootflags=subvol=@arch loglevel=3 quiet initrd /boot/intel-ucode.img^/@arch/boot/initramfs-linux-zen.img } Even tho it says that it found Fedora with the grub-mkconfig command it actually didn't even add it as a boot option in grub.cfg I can confirm that the last version of Fedora Workstation 37 does all this properly and there is no problems there for detecting and adding the entries of the OS's installed. I will post here the patches that they are using. https://src.fedoraproject.org/rpms/os-prober/tree/f37 It seems that the patches they apply are what makes their packaged version of os-prober work properly.
(In reply to invra from comment #0) > Another problem is that it is adding a ^ initrd > /boot/intel-ucode.img^/@arch/boot/initramfs-linux-zen.img This requires a patched sys-boot/grub as per https://github.com/gentoo/gentoo/pull/20389. os-prober is patched and upstream grub has already accepted the commit with https://git.savannah.gnu.org/cgit/grub.git/commit/util/grub.d/30_os-prober.in?id=000b5cd04fd228f9741f5dca0491636bc0b89eb8 but grub hasn't released a new version since that time. In the meantime you can save https://git.savannah.gnu.org/cgit/grub.git/patch/util/grub.d/30_os-prober.in?id=000b5cd04fd228f9741f5dca0491636bc0b89eb8 to a file in /etc/portage/patches/sys-boot/grub, reinstall grub, and rebuild grub.cfg. > I can confirm that the last version of Fedora Workstation 37 does all this > properly and there is no problems there for detecting and adding the > entries of the OS's installed. Yes, any OS can recognize itself via grub without even resorting to os-prober. But as for booting Arch, Fedora should suffer the same issue as Gentoo as I don't see the os-prober/grub patches to fix these (in line with https://ask.fedoraproject.org/t/grub-triple-boot-doesnt-show-other-linux/15901/2). If I am in error, please attach Fedora's /boot/grub/grub.cfg and I'll take a look. Also, I don't know of a distro whose grub/os-prober can detect Fedora in a btrfs subvolume, besides the booted Fedora that is (eg., https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1967944). If I'm in error, please attach its /boot/grub/grub.cfg and I'll take a look.
I jsut applied the patch you posted and it fixes the ^ error but it still does not look right. linux /boot/vmlinuz-linux root=UUID=*** rw rootflags=subvol=@arch loglevel=3 quiet initrd /boot/intel-ucode.img /@arch/boot/initramfs-linux-fallback.img It should look like this: linux /@arch/boot/vmlinuz-linux root=UUID=*** rw rootflags=subvol=@arch loglevel=3 quiet initrd /@arch/boot/intel-ucode.img /@arch/boot/initramfs-linux-fallback.img For some reason it is not picking up the subvolume. It doesn't make any sense how the initramfs is /@arch/boot/initramfs-linux-fallback.img but /boot/intel-ucode.img is missing @arch. Same goes for the kernel line, there is no @arch.
Created attachment 850296 [details, diff] os-prober-btrfs-testing-v1.patch Could you try adding this patch to /etc/portage/patches/sys-boot/os-prober/ and see if that resolves the issue?
Did some independent testing. OS-Prober needs to use `subvolid` and not `subvol`(e.g. the numbers not the path names). If you have mounted a subvolume directly then the pathnames are relative to that mount point subvolume. Consequently these relative paths are invalid in subsequent mount commands. Consider mt btrfs volume: /Subvoumes/__{System,Home} that are mounted as mount -t btrfs -o subvol=/Subvolume/__System /dev/nvme0n1p3 / mount -t btrfs -o subvol=/Subvolume/__Home /dev/nvme0n1p3 /home Now imagine I do a `btrfs sub create /usr/src/local_kernel` btrfs sub list with give subvol=usr/src/local_kernel os-prober will then try mount -t btrfs -o subvol=usr/src/local_kernel /dev/nvme0n1p3 /wherever The almost impossible-to-determine correct command would be mount -t btrfs -o subvol=/Subvolume/__System/usr/src/local_kernel /dev/nvme0n1p3 /wherever And if I make a `btrfs sub create ~/somewhere` the disambiguation happens twice if you want the correct result. Who has time for that. The numeric subvolume ids, however, are global, unique, available in the second column of `btrfs sub list` with no need to cook. Presuming 251 is the correct id in question mount -t btrfs -o subvolid=251 /dev/nvme0n1p3 /wherever will do the right thing in all cases. As a side note, a failed mount should _not_ stop the process of searching,IMHO, it should be treatead as a negative outcome and the process should move on. ASIDE: Why do all my systems do that /Subvolume/__System thing? Simple. You take a snapshot of the root volume into a subdirectory, but you cannot use that snapshot to restore the restore/replace root volume in total, you can only copy over it file by file. this is not good for removing improperly added files etc. In the above system I can (and have) restored a system with trivial downtime by booting my emergency boot media and doing: mount -t btrfs -o subvol=/ /dev/nvme0n1p3 /mnt. mv /mnt/Subvolume/__System /mnt/Triage/ btrfs sub snap /mnt/Snapshots/__System_date /mnt/Sobvolume/__System reboot Also you can easily use btrfs send and receive to move snapshots to and from backup media, which is good for obvious reasons. Also Also its great for diskless nfs boots because of instant snapshottery being NFS4 friendly. etc. You can also also alos move the backup into place and make it read-write, but that burns up that snapshot so if the error happens again the snapshot is forever lost; so better to make the new snapshot. (All of boot is in my EFI partition so even that is safe. I mention this whole aside to explain why this sort of thing is likely to crop up in the wild for larger systems. IMPORTANT NOTE: subvolid does suck for the actual root= commandline stuff because the number sticks with the snapshot so movint it a al so the /mnt/Triage move is "harder" is problematic. User is recommended to use the default subvolume feature for the master __System and remember to reset it after the move-and-replace, leaving subvol{id}= off the master default boot command line in grub etc. (Yea, I poked this bear hard. ha ha ha.)
The problem is that what ens up calling linux-boot-prober is /etc/grub.d/30_os-prober from sys-boot/grub. os-prober, being a collection of unintuitive "spaghetti" shell code that's hard to reason about, is considered (rightfully) unsafe. Given that os-prober is disabled by default in grub, I doubt that upstream is interested in adding subvolid functionality to it. And the burden of the downstream sys-boot/grub devs to lug around a patch to enable changes that break the expected behavior of both upstream packages is huge ask. I would sooner look in to changing the dependence on `btrfs sub list` in /usr/lib/os-probes/50mounted-tests.
Upon further inspection, it doesn't look like the use of `btrfs subvolume list` is problematic at all. os-prober only uses it to compare with the output of `btrfs subvolume list -r` and `btrfs subvolume list -s`, to exclude read-only subvolumes and snapshots from other subvolumes. The rest of /usr/lib/os-probes/50mounted-tests depends heavily on the output of `grep btrfs /proc/self/mountinfo` and I can see no other reference to `btrfs subvolume list' in os-prober.
Created attachment 868957 [details, diff] os-prober-btrfs-testing-v2.patch So it appears the subvolumes from `btrfs subvolume list`, after being filtered, are used to parse the output of /proc/self/mountinfo (eg., `grep "/$subvol "`). Without having to overcomplicate things further, it seems a valid absolute path to the subvolume can be had by adding the -a flag to the `btrfs subvolume list` command and removing any potential instances of '<FS_TREE>/'. Please test.
