I have installed ``` sys-kernel/linux-firmware-20240312::gentoo USE="compress-zstd initramfs redistributable -bindist -compress-xz -deduplicate -savedconfig (-unknown-license)" ``` savedconfig is ``` voyager /etc/portage/savedconfig/sys-kernel # cat linux-firmware-20240312 amdgpu/navy_flounder_sos.bin amdgpu/navy_flounder_ta.bin amdgpu/navy_flounder_smc.bin amdgpu/navy_flounder_dmcub.bin amdgpu/navy_flounder_pfp.bin amdgpu/navy_flounder_me.bin amdgpu/navy_flounder_ce.bin amdgpu/navy_flounder_rlc.bin amdgpu/navy_flounder_mec.bin amdgpu/navy_flounder_mec2.bin amdgpu/navy_flounder_sdma.bin amdgpu/navy_flounder_vcn.bin amdgpu/yellow_carp_toc.bin amdgpu/yellow_carp_ta.bin amdgpu/yellow_carp_dmcub.bin amdgpu/yellow_carp_pfp.bin amdgpu/yellow_carp_me.bin amdgpu/yellow_carp_ce.bin amdgpu/yellow_carp_rlc.bin amdgpu/yellow_carp_mec.bin amdgpu/yellow_carp_mec2.bin amdgpu/yellow_carp_sdma.bin amdgpu/yellow_carp_vcn.bin regulatory.db regulatory.db.p7s ath11k/WCN6855/hw2.1/firmware-2.bin ath11k/WCN6855/hw2.1/amss.bin ath11k/WCN6855/hw2.1/board-2.bin ath11k/WCN6855/hw2.1/m3.bin qca/rampatch_usb_00130201.bin ``` files exist (compressed), but some are symlinks. ``` voyager /etc/portage/savedconfig/sys-kernel # file /usr/lib/firmware/ath11k/WCN6855/hw2.1/board-2.bin.zst /usr/lib/firmware/ath11k/WCN6855/hw2.1/board-2.bin.zst: symbolic link to ../hw2.0/board-2.bin.zst ``` re-emerging with USE="savedconfig" breaks the symbolic links, which are then later cleaned up. ``` >>> Install sys-kernel/linux-firmware-20240312 into /var/tmp/portage/sys-kernel/linux-firmware-20240312/image copying/compressing file amdgpu/navy_flounder_ce.bin copying/compressing file amdgpu/navy_flounder_dmcub.bin copying/compressing file amdgpu/navy_flounder_me.bin copying/compressing file amdgpu/navy_flounder_mec.bin copying/compressing file amdgpu/navy_flounder_mec2.bin copying/compressing file amdgpu/navy_flounder_pfp.bin copying/compressing file amdgpu/navy_flounder_rlc.bin copying/compressing file amdgpu/navy_flounder_sdma.bin copying/compressing file amdgpu/navy_flounder_smc.bin copying/compressing file amdgpu/navy_flounder_sos.bin copying/compressing file amdgpu/navy_flounder_ta.bin copying/compressing file amdgpu/navy_flounder_vcn.bin copying/compressing file amdgpu/yellow_carp_ce.bin copying/compressing file amdgpu/yellow_carp_dmcub.bin copying/compressing file amdgpu/yellow_carp_me.bin copying/compressing file amdgpu/yellow_carp_mec.bin copying/compressing file amdgpu/yellow_carp_mec2.bin copying/compressing file amdgpu/yellow_carp_pfp.bin copying/compressing file amdgpu/yellow_carp_rlc.bin copying/compressing file amdgpu/yellow_carp_sdma.bin copying/compressing file amdgpu/yellow_carp_ta.bin copying/compressing file amdgpu/yellow_carp_toc.bin copying/compressing file amdgpu/yellow_carp_vcn.bin copying/compressing file qca/rampatch_usb_00130201.bin creating link ath11k/WCN6855/hw2.1/board-2.bin -> ../hw2.0/board-2.bin creating link ath11k/WCN6855/hw2.1/amss.bin -> ../hw2.0/amss.bin creating link ath11k/WCN6855/hw2.1/m3.bin -> ../hw2.0/m3.bin * Removing broken symlinks ... ath11k/WCN6855/hw2.1/board-2.bin ath11k/WCN6855/hw2.1/amss.bin ath11k/WCN6855/hw2.1/m3.bin ``` To work-around, one needs to also list the symlink targets in the saved config, but this is tedious. Perhaps this could be mentioned in the linux-firmware wiki? I dont know what the ideal solution is...? A) dont use symlinks when savedconfig is enabled? B) use hardlinks instead of symlinks ? C) try to be clever, and also preserve link targets?
Any progress? Thanks.
Created attachment 908359 [details, diff] Possible fix for matching symlinks, not their targets I ran into the same problem. My preference is to have the Gentoo savedconfig modification to copy-firmware.sh check for symbolic links themselves instead of their targets. This patch is a proposed modification to copy-firmware.sh to do just that. It also switches to using bash associative arrays instead of grep for detecting matches, avoiding issues with treating the periods in path names as regular expression wildcards and calling out to an external program, at the cost of requiring bash instead of just any POSIX sh-compatible shell. The modifications are relatively minor, and if taken, I would recommend including them in a new version of the existing copy-firmware.sh patch. I wrote the modifications myself, and submit them as abandonware, with no owner, free to use for anybody who wants to use them, for whatever purpose they wish. I realize might not conform to GLEP 76, but I hope they are still usable as I consider this a small enough modification that it is not copyrightable.
Created attachment 908361 [details] Proposed ebuild modification for above patch This is a very basic modification to the existing ebuild that applies the above patch, for anybody who wants to apply it with a modified ebuild in an overlay or something. Alternatively, the above patch could be applied by placing it in /etc/portage/patches/.
Created attachment 908362 [details, diff] Alternative to possible fix for matching symlinks, not their targets This is an alternative to my earlier patch to copy-firmware.sh. It also checks for symlinks instead of their targets, except does so with near-minimal modifications. It adds some flags to the existing grep call to avoid confusion with periods in paths being treated as regular expression wildcards, but is probably a little more prone to grep interpreting the argument incorrectly than the other version. Unlike my other version, it does not use bash associative arrays, and so should be compatible with POSIX sh-compatible shells. This modification is tiny enough that it really cannot be considered copyrightable. It should work as a drop-in replacement for my earlier patch (as in, it should work with the ebuild modification I submitted, or dropped into /etc/portage/patches/, using the same name). The two patches should not be used together; they will conflict as the modify the same lines. Either one will hopefully work by itself.
I tried linux-firmware-20241017-r3.ebuild with both patches and it doesn't install the symlink targets. It just removes the symlinks.
(In reply to Anon Emuss from comment #2) > Created attachment 908359 [details, diff] [details, diff] > Possible fix for matching symlinks, not their targets > > I ran into the same problem. My preference is to have the Gentoo > savedconfig modification to copy-firmware.sh check for symbolic links > themselves instead of their targets. This patch is a proposed modification > to copy-firmware.sh to do just that. It also switches to using bash > associative arrays instead of grep for detecting matches, avoiding issues > with treating the periods in path names as regular expression wildcards and > calling out to an external program, at the cost of requiring bash instead of > just any POSIX sh-compatible shell. Just so as you know, I prepared a patch that took a very similar approach shortly before the breakage was realised that lead to bug 925297 being filed back in February. It was rejected on the basis that the prevailing approach is upstreamable (I rather think not) and that to use bash would stymie any such effort.
Apologies; I appear to have misread this report. I ran into a problem where specifying both the source and the target of symlinks sometimes did not work, because the copy-firmware.sh script recognized symlinks by their target, treated as a regular expression, which is frequently something like ../../sourcedir1/sourcedir2/source. I set up my saved configuration by manually clearing out everything I did not want, and then listing the contents of what was left. With that, symlink targets and sources were included, but they were specified as linkdir1/linkdir2/symboliclink and sourcedir1/sourcedir2/source, neither of which match the regular expression ../../sourcedir1/sourcedir2/source, and so the symbolic link would not get included. The patches I provided fix this problem, but still require both the symbolic link and the target of the link to be specified. The patches I provided do not fix the problem (or maybe a feature request?) reported here.
Created attachment 913789 [details, diff] Patch for copy-firmware.sh to support symlinks. Here, a patch for copy-firmware.sh. Files that are symlinks can be specified in the savedconfig file with this patch. It is a replacement for the existing patch. Uses `grep -q` for matching entries as in the current patch.
Tested. Works. Thank you!
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=937772802f929b3cfee5e494508c7ff539594067 commit 937772802f929b3cfee5e494508c7ff539594067 Author: Mike Pagano <mpagano@gentoo.org> AuthorDate: 2024-12-17 17:42:37 +0000 Commit: Mike Pagano <mpagano@gentoo.org> CommitDate: 2024-12-17 17:42:37 +0000 sys-kernel/linux-firmware: Fixes for copyfirmware.sh Thanks to 5FBCDB43 Bug: https://bugs.gentoo.org/927022 Signed-off-by: Mike Pagano <mpagano@gentoo.org> .../files/linux-firmware-copy-firmware-r7.patch | 57 +++ .../linux-firmware-20241210-r1.ebuild | 406 +++++++++++++++++++++ 2 files changed, 463 insertions(+)
(In reply to 5FBCDB43 from comment #8) > Created attachment 913789 [details, diff] [details, diff] > Patch for copy-firmware.sh to support symlinks. > > Here, a patch for copy-firmware.sh. > > Files that are symlinks can be specified in the savedconfig file with this > patch. It is a replacement for the existing patch. > > Uses `grep -q` for matching entries as in the current patch. Thanks everyone for the work and thanks 5FBCDB43 for the patch. This is now in linux-firmware-20241210-r1. Before we release to the masses please confirm we are all good here.
Tested linux-firmware-20241210-r1 today with USE="redistributable savedconfig -bindist -compress-xz -compress-zstd -deduplicate -dist-kernel -initramfs (-unknown-license)". Works OK. Thank you!
(In reply to Marius Dinu from comment #12) > Tested linux-firmware-20241210-r1 today with USE="redistributable > savedconfig -bindist -compress-xz -compress-zstd -deduplicate -dist-kernel > -initramfs (-unknown-license)". Works OK. Thank you! Thank-you for this testing. I appreciate it.
Hello, the patch has broke again
