Some distributions already support a straightforward (less that five minutes if it's not your first time) way to boot their installation images from network. They usually don't rely on having the same distribution on the server (for example I'm using OpenWRT). Expected results: Images for network booting Gentoo Installation live system available over HTTP from the official archive and mirrors. Two or three images are typically used (although it's possible to use a single image as well): * kernel * initramfs - must get you connected over ethernet * squashfs (not needed if initramfs contains everything) Typically, those three tools are downloaded to the boot server. The bootloader (not included in the list) is served via tftp (using a PXE ROM) and it in turn uses the PXE stack to download kernel and initramfs. When the kernel initializes itself using the initramfs, it can already use HTTP to download the squashfs image. For embedded devices with little storage, it's better to first boot an extended PXE firmware like iPXE (formerly gPXE) which is capable of working over HTTP instead of tftp. Then you can serve anything (or everything) directly from the mirrors, without the need to download the tools locally. * iPXE image to be downloaded to the boot server * pxelinux.0 (syslinux) * menu.c32 (syslinux) * pxelinux.cfg/default (custom configuration file) * kernel (live system) * initramfs (live system) * squashfs (live system, optional) Actual results: AFAIK Gentoo used to support the use case but doesn't any more. Old documentation doesn't work. I'll add any source information I find but so far I have: http://ercpe.de/blog/how-to-boot-a-gentoo-livecd-via-pxe Proposal: I have a couple of steps in mind that could work for us: 1) Improve the initramfs distributed on live CDs to also support the non-CD use case... The following patch might help to identify the issues. https://gist.github.com/bondario/2762717 2) Also distribute kernel, initramfs and squashfs separately, to avoid the need to download and unpack the live CD image. 3) Also distribute the ipxe/syslinux stuff to help users minimize the boot system. Known other systems: * boot.kernel.org: distributed such images for numerous linux distributions before its close down. * boot.fedoraproject.org: distributes images to boot Fedora over Internet. Please let me know if I can provide any help.
It would likely help a lot to join #gentoo-releng on irc.freenode.net and discuss it with the team. I like the idea, but honestly I don't think any of us have the time to do this without significant help and it would be foolish to ask you to write patches for us without discussing implementation details first.
BUG #396467 has patch to make the genkernel init be able to detect squashfs in the intird No need for syslinux/pxelinux, iPXE can do the patching on the fly (with a few gotchas for EFI) To fully support this the files needed are the kernel, initrd, and squashfs that is normally placed on the isos. Having these extracted outside the isos on the mirrors would be nice from a netbooting perspective. But it would take some space. Other than that we could provide and document the process in an iPXE script, see above mentioned bug for example.
Continuing on from comment #2 I'm trying to spend some time to get this further. Currently in the process of trying to get easy to use and understand examples up at https://github.com/NiKiZe/Gentoo-iPXE and having an issue open about the needed files being provided on Gentoo mirrors without having to download and unpack parts of the iso. https://github.com/NiKiZe/Gentoo-iPXE/issues/2 In that issue it is documented what we want to have on the mirrors: * boot/gentoo - the kernel * boot/gentoo.igz - initrd * image.squashfs * boot.ipxe - iPXE boot script - this is a maybe The files above (except for the script) is already in the iso - and does now work (at least for me) without any modifications for network boot. If this is an interesting way forward, then what I propose is that the mentioned kernel, initrd, squash that now exists only in the minimal iso - is also made available as separate files on the mirrors.
PoCs to generate needed files One variant modifies upload step https://github.com/NiKiZe/releng/commit/8408f98 Another variant modifies ISO creation https://github.com/NiKiZe/catalyst/commit/33c9405
(In reply to Christian Nilsson from comment #4) > PoCs to generate needed files > One variant modifies upload step > https://github.com/NiKiZe/releng/commit/8408f98 > Another variant modifies ISO creation > https://github.com/NiKiZe/catalyst/commit/33c9405 Hi Christian, this is pretty cool, and definitely interesting. Let's clean it up a bit and maybe we can merge it then. First thing... could you change your catalyst steps so you don't modify the iso target in catalyst but add a new target instead? Like pxeboot maybe? Don't worry too much about building things twice for iso and pxeboot, catalyst uses binary package caching a lot. You can depend on other targets, but first packing everything into an iso and then unpacking it makes no sense. Once this is done, we can think about the uploading part.
Very nice work. This is great. I think we should wire this into catalyst's "netboot" target, which already exists for some platforms (mips, sparc, hppa, etc) and of course has to do hardware-specific things.
Thank you for you replies. And sorry for this wall of text. The 2 PoCs provided so far is xor. (stated in the commits) Also worth mentioning that the way that these are done right now is actually in use in the wild. If you have qemu installed, please give this a whirl: > qemu-system-x86_64 -enable-kvm -M q35 -m 1024 -cpu host -nic user,model=virtio,bootfile=http://b800.org/gentoo/boot.ipxe -boot menu=on -usb And to keep the eyes on the target (for me) it is to be able to have it working with this instead: > ,bootfile=https://distfiles.gentoo.org/releases/amd64/autobuilds/current-install-amd64-minimal/install-amd64-minimal-latest.ipxe I strongly think that this should be based on the livecd. There is no reason to have different kernels/initrd/squashfs Instead by using the same files it is less likely that there is a disconnect and that one work while the other don't, or that the loved livecd get it's packet list updated while the pxeboot target don't etc. I'm not at all worried about building twice, but rather about the 2 being the same for verification and usage reasons. This "new way" of PXE boot does not have any hardware specific things, and so IMO netboot is the wrong thing here. Might misunderstand tho, I know netboot are great for the other arches, proposed version should be valid for x86|amd64|arm* which has working iPXE implementation (supports http based netboot). For a long time the way users has done PXE-boot for amd64 in Gentoo is to modify the minimal iso, I recently modified the wiki, but even before then only minimal modifications was made. (mentioned the qemu testcase above) Did consider about something like the below, but my catalyst target fuu/understanding is to small for me right now. (in some places even modifying catalyst itself) * installcd-stage2.spec - build squashfs, kernel, initrd. * installcd-stage3-minimal.spec - add bootloaders (current grub, syslinux), and generate iso. * installpxe-stage3-minimal.spec - repackage initrd+squashfs and generate config files/scripts This probably makes to much of a disconnect between settings in the stage2 spec files and what would be needed for stage3. - not to mention all the places and documentation that would need updates in regards to livcd stage2/3. Early on I did write this up, but it's not up to date. https://github.com/NiKiZe/Gentoo-iPXE/issues/2#issuecomment-886102757 Another way could be to have installpxe-stage3-minimal.spec that actually did the steps that the releng upload commit does. aka unpacking the ISO, it might not be worse unpacking the iso than it is to unpack any tgz/tar.xz After thinking about this several times over to get it cleaner, splitting the .ipxe script generation into being next to syslinux/grub config generation, and then adding conditionals for when this extra "iso variant" would be created. (this would result in .ipxe file sitting on iso, but that cost is minimal at most) One freestanding question tho, is there anything in regards to DIGESTS that needs to be handled as well?
Not a full response -- just a quick question: (In reply to Christian Nilsson from comment #7) > There is no reason to have different kernels/initrd/squashfs Are there any size limitations to PXE images? That is, are there cases where a PXE image won't boot if it's too large? Netboot images on other platforms are often extremely size constrained, which is probably why they're an entirely different target in catalyst.
(In reply to Matt Turner from comment #8) > Not a full response -- just a quick question: > > (In reply to Christian Nilsson from comment #7) > > There is no reason to have different kernels/initrd/squashfs > > Are there any size limitations to PXE images? That is, are there cases where > a PXE image won't boot if it's too large? > > Netboot images on other platforms are often extremely size constrained, > which is probably why they're an entirely different target in catalyst. We should go back a bit to explain this. PXE (legacy pcbios) needs to fit in 640KB, this is together with other things that might already exist in that memory space. iPXE, PXELINUX, GRUB must be loaded in this space. efi has less of an issue. Other limitations are TFTP max size (4GB with the right extensions) Other than this it is RAM - which is less of an issue these days. As an example, the qemu testcase has 1GB RAM, the full initrd image is 390MB With less RAM than 1GB it fails (black screen) or takes long time to boot. So yes, there is constraints which is why solutions such as kernel+initrd is loaded and then squashfs is connected over NFS - and this is needed in the mentioned low memory situations. But it adds complexity, not least on infrastructure, and this is why I think that this new PXE approach for modern (x86 based) machines is the simplest to get running. So I will modify my previous statement a bit, netboot might be interesting for x86* as well, but that is a different beast from what my solution. And I think it should be kept different from the "pxeboot" idea. Instead of calling it any net/pxe related - the basic need (from iPXE users) is that the kernel, initrd, squashfs be hosted on http outside the iso. I just added on that and made it usable for PXELINUX and grub as well.
