Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 824570

Summary: dev-util/catalyst-3.0.20 "No space left on device. Failed to populate EFI image file"
Product: Gentoo Hosted Projects Reporter: Dmitrii Aleksandrov <adk230>
Component: CatalystAssignee: Gentoo Catalyst Developers <catalyst>
Status: UNCONFIRMED ---    
Severity: normal CC: bkohler
Priority: Normal    
Version: unspecified   
Hardware: AMD64   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: output of emerge --info dev-util/catalyst

Description Dmitrii Aleksandrov 2021-11-18 17:43:41 UTC
Created attachment 752634 [details]
output of emerge --info dev-util/catalyst

I'm using catalyst-3.0.20 to build amd64/installcd-stage2-minimal.spec from releng repository. During "create_iso" action, it creates a loopback file, then tries to populate the image and fails because it runs out of space in that file:

18 Nov 2021 20:29:51 MSK: NOTICE  : --- Running action sequence: create_iso
NOTICE:catalyst:--- Running action sequence: create_iso
Preparing EFI boot image
Creating loopback file of size 10766kB
10766+0 records in
10766+0 records out
11024384 bytes (11 MB, 11 MiB) copied, 0.0377808 s, 292 MB/s
Formatting loopback file with FAT FS
mkfs.fat 4.2 (2021-01-31)
Mounting FAT loopback file
Populating EFI image file from /var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/boot/EFI
'/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/boot/EFI/' -> '/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/gentoo.efimg.mountPoint/EFI'
'/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/boot/EFI/BOOT' -> '/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/gentoo.efimg.mountPoint/EFI/BOOT'
'/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/boot/EFI/BOOT/grubia32.efi' -> '/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/gentoo.efimg.mountPoint/EFI/BOOT/grubia32.efi'
'/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/boot/EFI/BOOT/BOOTIA32.EFI' -> '/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/gentoo.efimg.mountPoint/EFI/BOOT/BOOTIA32.EFI'
'/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/boot/EFI/BOOT/mmia32.efi' -> '/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/gentoo.efimg.mountPoint/EFI/BOOT/mmia32.efi'
'/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/boot/EFI/BOOT/grubx64.efi' -> '/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/gentoo.efimg.mountPoint/EFI/BOOT/grubx64.efi'
'/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/boot/EFI/BOOT/BOOTX64.EFI' -> '/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/gentoo.efimg.mountPoint/EFI/BOOT/BOOTX64.EFI'
'/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/boot/EFI/BOOT/mmx64.efi' -> '/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/gentoo.efimg.mountPoint/EFI/BOOT/mmx64.efi'
cp: error writing '/var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/gentoo.efimg.mountPoint/EFI/BOOT/mmx64.efi': No space left on device
Failed to populate EFI image file
18 Nov 2021 20:29:51 MSK: ERROR   : CatalystError: cmd(['/usr/share/catalyst/targets/livecd-stage2/livecd-stage2-controller.sh', 'iso', '/var/tmp/catalyst/builds/benchmark/install-amd64-minimal-latest.iso']) exited 1

I've fixed this problem by editing /usr/share/catalyst/targets/create-iso.sh and adding a larger slack to iaSize at line 228 (changed 64 to 10000). I don't feel like proposing this change as a finished patch, because I've chosen that new slack value randomly. It very well may be unreasonably large or still too small for some users. Someone knowledgeable should inspect that action and make a meaningful change.

Please let me know if I need to provide any additional info, this is my first bug report.
Comment 1 Ben Kohler gentoo-dev 2021-11-18 17:45:42 UTC
Can you see just how much data is in /var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/boot/EFI/ and compare that to the 10766kB loopback image size that was calculated/created?
Comment 2 Dmitrii Aleksandrov 2021-11-19 10:00:16 UTC
(In reply to Ben Kohler from comment #1)
> Can you see just how much data is in
> /var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/boot/EFI/
> and compare that to the 10766kB loopback image size that was calculated/created?

$ du -s /var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/boot/EFI/
10712   /var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/boot/EFI/

$ ls -l /var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/boot/EFI/
total 0
drwxr-xr-x 1 root root 130 Nov 18 20:25 BOOT

$ ls -l /var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/boot/EFI/BOOT/
total 10712
-rw-r--r-- 1 root root  975536 Nov 18 20:25 BOOTIA32.EFI
-rw-r--r-- 1 root root 1210776 Nov 18 20:25 BOOTX64.EFI
-rw-r--r-- 1 root root 3305472 Nov 18 20:25 grubia32.efi
-rw-r--r-- 1 root root 3379200 Nov 18 20:25 grubx64.efi
-rw-r--r-- 1 root root  927824 Nov 18 20:25 mmia32.efi
-rw-r--r-- 1 root root 1159560 Nov 18 20:25 mmx64.efi
Comment 3 Dmitrii Aleksandrov 2021-11-19 10:42:05 UTC
(In reply to Ben Kohler from comment #1)

Looks like the calculated image size is 54kB greater than the data. Could it be that it's not enough because of FAT overhead or something else?
Comment 4 Ben Kohler gentoo-dev 2021-11-19 11:56:34 UTC
can you check with --apparent-size as well? That's what we're using to calculate the image size, and that's the actual size without the *current* filesystem overhead & other factors involved.
Comment 5 Dmitrii Aleksandrov 2021-11-19 12:52:11 UTC
With --apparent-size it's 10702. Which makes sense. It's 64kB (the current slack) less than 10766 (the image).
Comment 6 Ben Kohler gentoo-dev 2021-11-19 18:38:45 UTC
I'm not seeing the failure here but my efi image data is slightly smaller: "Creating loopback file of size 8166kB"

After increasing slack to 1000 (kilobytes), can you loopback mount that image and see exactly how kilobytes are used/free/total in "df" output?  Mine has only a bit of space left:

Filesystem      1K-blocks       Used  Available Use% Mounted on
/dev/loop0           8130       8104         26 100% /mnt/test

Could you also see how many files & directories exist on that image?  Maybe that is where the additional overhead comes from.
Comment 7 Dmitrii Aleksandrov 2021-11-23 09:10:04 UTC
Sorry for not responding, my stage1 and stage2 data got overwritten by a newer build. When I have time, I'll redo the old build and post here
Comment 8 Dmitrii Aleksandrov 2021-12-17 12:55:16 UTC
With 1000 kb slack:

$ sudo mount -t vfat -o loop /var/tmp/catalyst/builds/benchmark/livecd-stage2-amd64-latest/gentoo.efimg /home/dima/mountPoint

$ df mountPoint/
Filesystem     1K-blocks  Used Available Use% Mounted on
/dev/loop0         11654 10708       946  92% /home/dima/mountPoint

$ ls -lA mountPoint/
total 2
drwxr-xr-x 3 root root 2048 Dec 17 15:10 EFI

$ ls -lA mountPoint/EFI/
total 2
drwxr-xr-x 2 root root 2048 Dec 17 15:10 BOOT

$ ls -lA mountPoint/EFI/BOOT/
total 10704
-rwxr-xr-x 1 root root  975536 Dec 17 15:10 BOOTIA32.EFI
-rwxr-xr-x 1 root root 1210776 Dec 17 15:10 BOOTX64.EFI
-rwxr-xr-x 1 root root 3305472 Dec 17 15:10 grubia32.efi
-rwxr-xr-x 1 root root 3375104 Dec 17 15:10 grubx64.efi
-rwxr-xr-x 1 root root  927824 Dec 17 15:10 mmia32.efi
-rwxr-xr-x 1 root root 1159560 Dec 17 15:10 mmx64.efi

$ du --apparent-size mountPoint/EFI/BOOT/*
953     mountPoint/EFI/BOOT/BOOTIA32.EFI
1183    mountPoint/EFI/BOOT/BOOTX64.EFI
3228    mountPoint/EFI/BOOT/grubia32.efi
3296    mountPoint/EFI/BOOT/grubx64.efi
907     mountPoint/EFI/BOOT/mmia32.efi
1133    mountPoint/EFI/BOOT/mmx64.efi


I should also mention that stage1 and stage2 specs are not "as is" in the releng repo, but slightly modified for my needs at work. I should've provided these for reproducibility, but you wouldn't be able to build them, because stage2 uses an fsscript that installs some proprietary packages from my company's overlay. What I can do though, is:
* Check if my issue persists when building "vanilla" specs from releng.
* If not, then build my modified specs without executing that fsscript, and then provide them to you if the issue persists.
Comment 9 Ben Kohler gentoo-dev 2021-12-17 13:03:31 UTC
Your grub images are much larger than mine.  Can you share your "emerge -pv grub" output since catalyst is using the system grub copy?

I'm wondering if we should simply make the slack space scale linearly with total image size.
Comment 10 Dmitrii Aleksandrov 2021-12-17 13:40:05 UTC
$ emerge -pv grub

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] sys-boot/grub-2.06-r1:2/2.06-r1::gentoo  USE="fonts nls sdl themes truetype -device-mapper -doc -efiemu -libzfs -mount (-test)" GRUB_PLATFORMS="efi-32 efi-64 pc -coreboot -emu -ieee1275 -loongson -multiboot -qemu -qemu-mips -uboot -xen -xen-32 -xen-pvh" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB
Comment 11 Ben Kohler gentoo-dev 2021-12-17 18:55:40 UTC
I tested with disabling compression on catalyst's grub-mkstandalone call, my resulting EFI image was even larger than yours, 13946kB, and it still didn't fail.

So I'm still not sure why your EFI image is losing more space to overhead than mine.  Can you share "emerge -pv dosfstools" ?
Comment 12 Dmitrii Aleksandrov 2021-12-17 19:13:07 UTC
I'll post it on monday
Comment 13 Dmitrii Aleksandrov 2021-12-20 09:17:50 UTC
$ emerge -pv dosfstools

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] sys-fs/dosfstools-4.2::gentoo  USE="iconv -compat -test" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB