Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 866959 - live media: work-in-progress saved/persistent boot implementation
Summary: live media: work-in-progress saved/persistent boot implementation
Status: UNCONFIRMED
Alias: None
Product: Gentoo Release Media
Classification: Unclassified
Component: All ISO (show other bugs)
Hardware: AMD64 Linux
: Normal enhancement (vote)
Assignee: Gentoo Release Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-08-27 11:43 UTC by hairu
Modified: 2023-08-10 00:01 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
gentoo.igz method patch (initrd.000002.patch,2.41 KB, patch)
2022-08-27 11:44 UTC, hairu
Details | Diff
tentative genkernel method patch (initrd.genkernel.patch,2.45 KB, patch)
2022-08-27 11:45 UTC, hairu
Details | Diff
Script generating SquashFS image of changes to the liveGUI (live-save-changes.sh,813 bytes, application/x-shellscript)
2022-08-27 11:45 UTC, hairu
Details
updated gentoo.igz method patch (initrd.000003.patch,3.61 KB, patch)
2022-08-29 00:11 UTC, hairu
Details | Diff
updated genkernel method patch (initrd.genkernel.000003.patch,3.67 KB, patch)
2022-08-29 00:12 UTC, hairu
Details | Diff
updated gentoo.igz method patch (initrd.000005.patch,3.77 KB, patch)
2022-08-29 03:47 UTC, hairu
Details | Diff
updated genkernel method patch (initrd.genkernel.000005.patch,3.84 KB, patch)
2022-08-29 03:47 UTC, hairu
Details | Diff
Script generating SquashFS image of changes to the liveGUI, now with update/bring-up capabilities against a new ISO (live-save-changes.sh,3.57 KB, text/plain)
2022-09-01 00:39 UTC, hairu
Details
updated gentoo.igz method patch (initrd.000009.patch,4.91 KB, patch)
2022-09-02 23:07 UTC, hairu
Details | Diff
updated genkernel method patch (initrd.genkernel.000009.patch,4.97 KB, patch)
2022-09-02 23:08 UTC, hairu
Details | Diff
Script generating SquashFS image of changes to the liveGUI, now with update/bring-up capabilities against a new ISO (live-save-changes.202209060227.sh,3.94 KB, text/plain)
2022-09-06 07:03 UTC, hairu
Details
Script generating SquashFS image of changes to the liveGUI, now with update/bring-up capabilities against a new ISO (live-save-changes.202209061644.sh,4.56 KB, text/plain)
2022-09-07 01:20 UTC, hairu
Details

Note You need to log in before you can comment on or make changes to this bug.
Description hairu 2022-08-27 11:43:02 UTC
I have quickly written a preliminary script over the last half-day generating a SquashFS image containing only the changes that I made to the liveGUI environment (excluding /home, which I chose to leave in a partition by including fstab in my changes), and patched 2 scripts from the genkernel initramfs, to enable overlayfs lowerdir stacking of said SquashFS image on top of the liveGUI's image.squashfs.

However, I'm running into an issue with the logic I wrote for when the isoboot= boot parameter is not in use. Perhaps someone could advise, or I can spend more time rewriting the findmediamount line of my patch next week, to provide a way to do this with and actual optical disc. Right now, my setup involves using GRUB to loopback the iso and then boot the kernel with isoboot= so I focused more on getting it working with this aspect of the patch.

If you could please take a look at the patch: should I just try to mount -t auto the overlayfs_persist_dev= boot parameter, or should I try to get findmediamount working the way I'm meant to?

Reproducible: Sometimes

Steps to Reproduce:
1. Boot the liveGUI, sync ::gentoo and install sys-fs/squashfs-tools

2. Follow this initramfs extraction example with the live medium's /boot/gentoo.igz knowing that the file is an xz compressed cpio archive [ https://wiki.gentoo.org/wiki/Custom_Initramfs#Extracting_the_cpio_archive ]

3. Patch extracted data with the initrd.000002.patch (alternatively, try to patch genkernel with initrd.genkernel.patch - untested, I just changed the filenames to reflect their location in the source code)

4. Recreate the cpio archive and compress it with `cd /path/to/extracted/cpio && find . -print0 | cpio --quiet --null -o -H newc --owner root:root --force-local --reproducible -F ../new.cpio && xz -e --check=none -z -f -9 -T 0 ../new.cpio` then save it somewhere GRUB can access and specify it in grub.cfg

5. Make whatever other modifications to your liveGUI environment you want, such as specifying a new /home in /etc/fstab

6. Run the script I wrote to compare changes to the original liveGUI environment, and save the file that was generated to somewhere mountable by the initramfs (I chose a location in the same place as my iso, which is why isoboot= works the way it's coded)

7. Boot using grub by editing this template menuitem to your needs:

menuentry 'Gentoo LiveGUI with changes' {
        #insmod ntfs
        #insmod ext4
        set iso=livegui.iso
        set isopath=(hdN,gptN)/$iso
        loopback loop $isopath
        set root=(loop)
        #linux /boot/gentoo root=/dev/ram0 init=/linuxrc overlayfs nodhcp looptype=squashfs loop=/image.squashfs cdroot
        linux /boot/gentoo root=/dev/ram0 init=/linuxrc overlayfs nodhcp looptype=squashfs loop=/image.squashfs cdroot=/dev/sdXN isoboot=$iso overlayfs_persist=persist.squashfs
        initrd (hdN,gptN)/gentoo.igz
}
Actual Results:  
Your saved changes should have been loaded on early boot, including the rc-scripts that you enabled. However, only try the overlayfs_persist_dev= parameter if you think you know how to fix the patch. the debug boot parameter is your friend.

Expected Results:  
Should work every time... Apologies for the rushed patch, but I was thinking it might be ready for at least trying it out.
Comment 1 hairu 2022-08-27 11:44:40 UTC
Created attachment 801478 [details, diff]
gentoo.igz method patch
Comment 2 hairu 2022-08-27 11:45:08 UTC
Created attachment 801481 [details, diff]
tentative genkernel method patch
Comment 3 hairu 2022-08-27 11:45:54 UTC
Created attachment 801484 [details]
Script generating SquashFS image of changes to the liveGUI
Comment 4 hairu 2022-08-29 00:11:51 UTC
Created attachment 801724 [details, diff]
updated gentoo.igz method patch

Cleaned a lot up and got docache working. Please test the way you boot. GRUB2 no longer needed...
Comment 5 hairu 2022-08-29 00:12:31 UTC
Created attachment 801727 [details, diff]
updated genkernel method patch
Comment 6 hairu 2022-08-29 03:47:31 UTC
Created attachment 801730 [details, diff]
updated gentoo.igz method patch

Third time's a charm?
Comment 7 hairu 2022-08-29 03:47:58 UTC
Created attachment 801733 [details, diff]
updated genkernel method patch
Comment 8 hairu 2022-09-01 00:39:12 UTC
Created attachment 802303 [details]
Script generating SquashFS image of changes to the liveGUI, now with update/bring-up capabilities against a new ISO

To save changes based on the currently-run ISO, just run the script.

To update/bring-up and save changes based on a new ISO, run the script with the UPDATE_TO=/path/to/livegui.iso environment variable... This is partly an interactive script: it will run etc-update once before installing/updating the packages that are in your modified /var/lib/portage/world, then it will run etc-update again before applying changes into the persistent image.

PS: Do not symlink your files if you are using the docache boot parameter! Also, docache seems to still be keeping the source block device(s) for image.squashfs and persist.squashfs mounted while booting when using overlayfs, so it is incomplete. I will see about fixing this tomorrow.
Comment 9 hairu 2022-09-02 23:07:24 UTC
Created attachment 802798 [details, diff]
updated gentoo.igz method patch

docache should be properly fixed for overlayfs. please note that entries/modifications to fstab involving the same block device(s) used to boot must use spaces rather than tabs for the time being. apologies, got very tired...
Comment 10 hairu 2022-09-02 23:08:21 UTC
Created attachment 802801 [details, diff]
updated genkernel method patch
Comment 11 hairu 2022-09-06 07:03:25 UTC
Created attachment 803362 [details]
Script generating SquashFS image of changes to the liveGUI, now with update/bring-up capabilities against a new ISO

I changed/improved the behaviour of the updating part of the script to avoid running emerge/smart-live-rebuild when they aren't installed on the live medium being updated (untested/not-working install/admin cd support, which would still require squashfs-tools to be installed on them, which it isn't), as well as added set -e to stop the script on errors.
Comment 12 hairu 2022-09-07 01:20:05 UTC
Created attachment 803641 [details]
Script generating SquashFS image of changes to the liveGUI, now with update/bring-up capabilities against a new ISO

Fixed some bugs, might still be problematic... I will conduct further tests soon.