in bug https://bugs.gentoo.org/212794 a code was added to skip device nodes for devices if at least one partition exists on the device. https://github.com/gentoo/genkernel/commit/cc57e07bb855e8f41e92b7d67098375587947693 so if both /dev/sdX and /dev/sdX1 exist, genkernel will always skip /dev/sdX device node and attempt to mount numbered partition. it works fine for x86/x64 boot media, because iso file partition table looks like this: > Device Boot Start End Sectors Size Id Type > install-amd64-minimal-20210613T214502Z.iso1 * 0 858111 858112 419M 0 Empty > install-amd64-minimal-20210613T214502Z.iso2 192 13259 13068 6.4M ef EFI (FAT-12/16/32) you see, partition 1 starts at sector 0, so it does not make a difference how you mount it, as /dev/sdX or /dev/sdX1, it's the same. same for x68 media. however, for powerpc/ppc64le media story is different > Device Boot Start End Sectors Size Id Type > install-powerpc-minimal-20210614T080607Z.iso1 * 1 728683 728683 355.8M cd unknown > Device Boot Start End Sectors Size Id Type > install-ppc64le-minimal-20210614T080607Z.iso1 * 1 1321079 1321079 645.1M cd unknown 1st partition starts at sector 1. but we need to mount it as /dev/sdX, not /dev/sdX1 > losetup -P -f install-ppc64le-minimal-20210614T080607Z.iso # -P is needed to scan for partitions > mount /dev/loop0 /mnt/t > mount: /mnt/t: WARNING: source write-protected, mounted read-only. ^ works > mount /dev/loop0p1 /mnt/t > mount: /mnt/t: wrong fs type, bad option, bad superblock on /dev/loop0p1, missing codepage or helper program, or other error. ^ does not work. but genkernel forces usage of /dev/sdX1, and boot fails, because we can't mount ppc/ppc64le media like this. we need to use full device. first sector ( 512 bytes ) contains partition table. so we can't skip it. can this code be either deleted or adjusted not to skip base device node on ppc/ppc64/ppc64le ? it only affects isoboot/findmediamount
back then the original bug filed, it was kernel 2.6, no EFI, and iso files used different partition layout and grub-mkrescue to make em bootable.
sorry last sentence got cut off. I meant nowadays we use gurb-mkrescue to make iso files bootable. back in the day it was probably ISOLINUX, but I don't know for sure.
ping, this is still breaking releng ppc builds. or should I consider patching in catalyst instead?
Created attachment 721330 [details, diff] genkernel-ppc64-iso.patch simple patch that just removes that codepath.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/releng.git/commit/?id=35826178e4096692dd254565c426a0eef8b0cc45 commit 35826178e4096692dd254565c426a0eef8b0cc45 Author: Georgy Yakovlev <gyakovlev@gentoo.org> AuthorDate: 2021-07-03 21:50:27 +0000 Commit: Georgy Yakovlev <gyakovlev@gentoo.org> CommitDate: 2021-07-04 20:48:13 +0000 releases/portage/isos: add genkernel patch for ppc64 since it's still not fixed in genkernel, we can carry a local patch. patch is conditional for ppc64 only, it will affect only ppc64le iso for now. ppc64 installcd is ppc, not ppc64. Bug: https://bugs.gentoo.org/796272 Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org> releases/portage/isos/env/sys-kernel/genkernel | 35 ++++++++++++++++ .../portage/isos/patches/genkernel-ppc64-iso.patch | 47 ++++++++++++++++++++++ 2 files changed, 82 insertions(+)
This patch is basically reverting https://gitweb.gentoo.org/proj/genkernel.git/commit/?id=cc57e07bb855e8f41e92b7d67098375587947693. 1) This patch needs testing. We need to be sure that it won't break other media. 2) Because it is reverting fix for bug 212794, I am expecting that it will actually cause a regression. Note that bug 212794 seems to be about converting ISO live media into USB live media.
I tested patch on ppc64le media, it produces expected results, both as usb image and iso image (yes I have tested it as CDROM) as I tried to explain above arguments mentioned in https://bugs.gentoo.org/show_bug.cgi?id=212794 are no longer relevant, as install media has different partition layout nowadays, where partition 1 starts at sector 0, so both paths are mountable. and partition 1 IS the partition that contains squashfs image genkernel looking for to perform cdboot. in the old days, 13 years ago, installcd iso used different partition layout. steps to confirm irrelevancy without building iso: grab latest iso from distfiles, amd64 one > losetup -f -P install-amd64-minimal-20210613T214502Z.iso > # test by partition > mount /dev/loop0p1 /mnt/t > ls /mnt/t/ > boot EFI gentoo.efimg grub image.squashfs isolinux livecd README.txt > mount /mnt/t/image.squashfs /mnt/s > bin boot dev etc home lib lib64 media mnt opt proc root run sbin sys tmp usr var > umount /mnt/s /mnt/t > # test by whole device > mount /dev/loop0p1 /mnt/t > boot EFI gentoo.efimg grub image.squashfs isolinux livecd README.txt > mount /dev/loop0 /mnt/t > bin boot dev etc home lib lib64 media mnt opt proc root run sbin sys tmp usr var as you see it works both ways on images that were not working 13 years ago, where we did not have EFI and if you check partition layout for x86 > Device Boot Start End Sectors Size Id Type > /dev/loop0p1 * 0 858111 858112 419M 0 Empty > /dev/loop0p2 192 13259 13068 6.4M ef EFI (FAT-12/16/32) you may see that EFI partition overlaps with first partition, it's embedded into it. that's how grub-mkrescue creates bootable efi images. hence problems described in https://bugs.gentoo.org/212794 are no longer relevant or exist. for original reporter squashfs was not mountable via full drive image mount, but was mountable with partition mount. my testing above shows that squashfs is now mountable via both methods.
in the second test there's copy-paste error ofc I was mounting /dev/loop0, not loop0p1 > /dev/loop0 on /mnt/t type iso9660 (ro,relatime,nojoliet,check=s,map=n,blocksize=2048) > /mnt/t/image.squashfs on /mnt/s type squashfs (ro,relatime)
also note it affects only cd/squash booting, not general booting. I don't think there are a lot of users of that feature besides releng/catalyst. so it's relatively safe to test by merging. it would not affect general public.
ping I'm working on arm64 boot media, it also affected by same problem. it should be mounted as top level device, but genkernel skips it. can we already get a patched genkernel please?
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/releng.git/commit/?id=2dfdd9d2784eb61c649051abe86bbe6cc2d14e0f commit 2dfdd9d2784eb61c649051abe86bbe6cc2d14e0f Author: Georgy Yakovlev <gyakovlev@gentoo.org> AuthorDate: 2021-07-26 04:19:37 +0000 Commit: Georgy Yakovlev <gyakovlev@gentoo.org> CommitDate: 2021-07-26 04:19:37 +0000 releases/portage/isos/env/sys-kernel/genkernel: apply on arm64 too Bug: https://bugs.gentoo.org/796272 Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org> releases/portage/isos/env/sys-kernel/genkernel | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
here's how boot looks if patched >> Genkernel 4.2.3 (2021-07-26 06:12:54 UTC). Linux kernel 5.10.52-gentoo-arm64 >> Activating udev ... >> Hint: Use scandelay[=seconds] if your live medium is slow and boot fails >> Loading keymaps Please select a keymap from the following list by typing in the appropriate name or number. Hit Enter for the default "us/43" US English keymap. 1 azerty 8 cf 15 es 22 is 29 no 36 sg 43 us 2 be 9 colemak 16 et 23 it 30 pl 37 sk-y 44 wangbe 3 bepo 10 croat 17 fi 24 jp 31 pt 38 sk-z 4 bg 11 cz 18 fr 25 la 32 ro 39 slovene 5 br-a 12 de 19 gr 26 lt 33 ru 40 trf 6 br-l 13 dk 20 hu 27 mk 34 se 41 ua 7 by 14 dvorak 21 il 28 nl 35 sf 42 uk << Load keymap (Enter for default): >> ... >> Keeping default keymap >> Making tmpfs for /newroot mknod: /newroot/dev/tty0: File exists >> Looking for the cdrom >> Attempting to mount media: /dev/sda >> Attempting to mount media: /dev/sda1 >> Attempting to mount media: /dev/sdb >> Attempting to mount media: /dev/sdb1 >> Attempting to mount media: /dev/sdc >> Media found on /dev/sdc >> Determining root device (trying /dev/sdc) ... >> Root device detected as /dev/sdc! >> Determining looptype ... >> Copying loop file for caching ... it mounts/wants /dev/sdc not /dev/sdcX
ping it breaks arm64 ppc64 ppc media, we carry a workaround patch for now. releng is primary user of squash boot regular use case is not affected. I've tested the changes, it's all good, on 4 arches (ppc ppc64 arm64 amd64) what else is needed?
My testing doesn't cover live media yet. So I have no environment to reproduce and work on this on my own. Like said, your proposed patch is *reverting* a previous fix which implies that when undoing the fix, we will cause a regression. From reading the code I believe that rewriting that logic would be the better solution. However, like said, I can't test such changes at the moment. If you believe the proposed patch is the correct solution, have tested everything (including the use case from the initial problem which caused this change) and will take full responsibility for this change, we can proceed. But I will not take responsibility for a change I am unable to test.
please let's proceed. if it breaks something on isoboot/squash - point finger at me, I'll happily investigate.
please can releng finally get a fixed version?
Created attachment 738103 [details, diff] git-formatted and signed-off patch
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/proj/genkernel.git/commit/?id=a3e1af34bb4dce30d99fdeca7b8217f89f219a01 commit a3e1af34bb4dce30d99fdeca7b8217f89f219a01 Author: Georgy Yakovlev <gyakovlev@gentoo.org> AuthorDate: 2021-09-08 02:04:16 +0000 Commit: Thomas Deutschmann <whissi@gentoo.org> CommitDate: 2021-09-08 14:00:08 +0000 initrd.scripts: don't skip top level devices with partitions ppc64 media should be mounted as /dev/sdX, not as /dev/sdX1 this loop was skipping /dev/sdX if /dev/sdX1 is present. Bug: https://bugs.gentoo.org/212794 Bug: https://bugs.gentoo.org/796272 Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org> Closes: https://bugs.gentoo.org/796272 Signed-off-by: Thomas Deutschmann <whissi@gentoo.org> defaults/initrd.scripts | 18 ------------------ 1 file changed, 18 deletions(-) Additionally, it has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/genkernel.git/commit/?id=a3e1af34bb4dce30d99fdeca7b8217f89f219a01 commit a3e1af34bb4dce30d99fdeca7b8217f89f219a01 Author: Georgy Yakovlev <gyakovlev@gentoo.org> AuthorDate: 2021-09-08 02:04:16 +0000 Commit: Thomas Deutschmann <whissi@gentoo.org> CommitDate: 2021-09-08 14:00:08 +0000 initrd.scripts: don't skip top level devices with partitions ppc64 media should be mounted as /dev/sdX, not as /dev/sdX1 this loop was skipping /dev/sdX if /dev/sdX1 is present. Bug: https://bugs.gentoo.org/212794 Bug: https://bugs.gentoo.org/796272 Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org> Closes: https://bugs.gentoo.org/796272 Signed-off-by: Thomas Deutschmann <whissi@gentoo.org> defaults/initrd.scripts | 18 ------------------ 1 file changed, 18 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/releng.git/commit/?id=dc40be83020cfc1c9539e8dcd6af0be51c2d3144 commit dc40be83020cfc1c9539e8dcd6af0be51c2d3144 Author: Georgy Yakovlev <gyakovlev@gentoo.org> AuthorDate: 2021-10-14 03:20:54 +0000 Commit: Georgy Yakovlev <gyakovlev@gentoo.org> CommitDate: 2021-10-14 03:23:57 +0000 releases/portage/isos/env/sys-kernel/genkernel: only apply to 4.2.3 genkernel 4.2.4 and later includes this patch https://gitweb.gentoo.org/proj/genkernel.git/commit/?id=a3e1af34bb4dce30d99fdeca7b8217f89f219a01 Bug: https://bugs.gentoo.org/796272 Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org> releases/portage/isos/env/sys-kernel/{genkernel => genkernel-4.2.3} | 0 1 file changed, 0 insertions(+), 0 deletions(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/releng.git/commit/?id=d03adad41e937c58379d6d0c240626f19097af30 commit d03adad41e937c58379d6d0c240626f19097af30 Author: Georgy Yakovlev <gyakovlev@gentoo.org> AuthorDate: 2021-11-01 04:36:49 +0000 Commit: Georgy Yakovlev <gyakovlev@gentoo.org> CommitDate: 2021-11-01 04:37:13 +0000 releases/portage/isos: remove genkernel workarounds in https://gitweb.gentoo.org/repo/gentoo.git/commit/sys-kernel/genkernel?id=0151b485d8a614b366cb15241f8cab4e4bd4b38d genkernel 4.2.5 got stable, which includes required functionality We can finally drop hacks. Bug: https://bugs.gentoo.org/796272 Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org> .../portage/isos/env/sys-kernel/genkernel-4.2.3 | 35 ---------------- .../portage/isos/patches/genkernel-ppc64-iso.patch | 47 ---------------------- 2 files changed, 82 deletions(-)
