Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 695616 - dev-util/catalyst is hard-coded to install app-misc/livecd-tools
Summary: dev-util/catalyst is hard-coded to install app-misc/livecd-tools
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Hosted Projects
Classification: Unclassified
Component: Catalyst (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Catalyst Developers
URL:
Whiteboard:
Keywords: PATCH
Depends on:
Blocks:
 
Reported: 2019-09-25 09:31 UTC by Kenneth G. Strawn
Modified: 2019-10-11 16:02 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Archive containing modified ebuilds that fix this bug (livecd-tools.tar.xz,1004 bytes, application/x-xz)
2019-09-25 09:31 UTC, Kenneth G. Strawn
Details
Requested patch per Mike Gilbert (app-misc-livecd-tools-systemd.patch,3.44 KB, patch)
2019-09-25 20:38 UTC, Kenneth G. Strawn
Details | Diff
Oops, forgot to add the flags to IUSE (0001-Forgot-to-add-the-flags-to-IUSE.patch,2.69 KB, patch)
2019-09-25 22:00 UTC, Kenneth G. Strawn
Details | Diff
Stage 1 specfile (livecd-stage1.spec,26.74 KB, text/x-rpm-spec)
2019-09-29 16:13 UTC, Kenneth G. Strawn
Details
Stage 2 specfile (livecd-stage2.spec,1.49 KB, text/x-rpm-spec)
2019-09-29 16:14 UTC, Kenneth G. Strawn
Details
fsscript (customize.sh,974 bytes, application/x-shellscript)
2019-09-29 16:15 UTC, Kenneth G. Strawn
Details
catalystrc (catalystrc,368 bytes, application/x-shellscript)
2019-09-29 16:16 UTC, Kenneth G. Strawn
Details
Updated fsscript (customize.sh,978 bytes, application/x-shellscript)
2019-09-29 16:24 UTC, Kenneth G. Strawn
Details
Build automation utility (autocatalyst.tar.xz,37.56 KB, application/x-xz)
2019-10-02 02:26 UTC, Kenneth G. Strawn
Details
Wrapper script (autocatalyst.sh,3.13 KB, text/x-shellscript)
2019-10-04 14:32 UTC, Kenneth G. Strawn
Details
fsscript (customize.sh,2.38 KB, text/x-shellscript)
2019-10-04 14:35 UTC, Kenneth G. Strawn
Details
fsscript (customize.sh,2.26 KB, text/x-shellscript)
2019-10-04 14:40 UTC, Kenneth G. Strawn
Details
Script to resolve keyword issues because Catalyst fails to (fix-keywords.sh,371 bytes, text/x-shellscript)
2019-10-04 14:46 UTC, Kenneth G. Strawn
Details
livecd-stage1.spec (livecd-stage1.spec,25.03 KB, text/x-rpm-spec)
2019-10-04 14:48 UTC, Kenneth G. Strawn
Details
Kernel configuration (livecd-stage2.config,139.14 KB, text/plain)
2019-10-04 14:49 UTC, Kenneth G. Strawn
Details
livecd-stage2.spec (livecd-stage2.spec,1.54 KB, text/x-rpm-spec)
2019-10-04 14:51 UTC, Kenneth G. Strawn
Details
Remove hard-coded livecd-tools (0001-livecd_stage1.py-drop-hard-coded-app-misc-livecd-too.patch,1.00 KB, patch)
2019-10-04 14:54 UTC, Mike Gilbert
Details | Diff
Remove hard-coded livecd-tools (0001-livecd_stage1.py-drop-hard-coded-app-misc-livecd-too.patch,1.04 KB, patch)
2019-10-04 14:59 UTC, Mike Gilbert
Details | Diff
Add app-misc/livecd-tools to all livecd-stage1 specs (0001-Add-app-misc-livecd-tools-to-all-livecd-stage1-specs.patch,7.97 KB, patch)
2019-10-04 15:35 UTC, Mike Gilbert
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Kenneth G. Strawn 2019-09-25 09:31:21 UTC
Created attachment 591016 [details]
Archive containing modified ebuilds that fix this bug

Whenever you try to use Catalyst to build bootable media, it always installs app-misc/livecd-tools, which, unnecessarily, tries to smuggle in openrc and, by extension, sysvinit whether you want it or not. That's a problem if you're working off of a systemd base and especially if you're looking to install a modern desktop system ― this may be the very reason why the current LiveDVD is from way back in 2016.

For this reason, I'm submitting a series of modified ebuilds (attached to this bug) that fix the problem by conditionally depending on openrc ONLY if the sysv-utils USE flag, while also conditionally depending on systemd if the systemd USE flag is set. This should have been done a long time ago.
Comment 1 Mike Gilbert gentoo-dev 2019-09-25 15:13:56 UTC
A tarball full of ebuilds is very difficult to review. Please submit a patch generated via "git format-patch".
Comment 2 Kenneth G. Strawn 2019-09-25 20:38:50 UTC
Created attachment 591092 [details, diff]
Requested patch per Mike Gilbert

Done as Mike requested ― here's the patch that changes the ebuild dependencies from hard-depends on openrc to USE-conditional dependencies that check whether the sysv-utils or systemd USE flag is set before proceeding.
Comment 3 Kenneth G. Strawn 2019-09-25 22:00:37 UTC
Created attachment 591096 [details, diff]
Oops, forgot to add the flags to IUSE

Fixing error that failure to add the systemd and sysv-utils flags to IUSE introduced when attempting to build the manifest
Comment 4 Kenneth G. Strawn 2019-09-27 11:35:53 UTC
Updated title to more accurately reflect the problem that my patch (which unfortunately still hasn’t been reviewed) is meant to fix.
Comment 5 Kenneth G. Strawn 2019-09-27 11:42:29 UTC
Per my patches — this should have been the case the whole time across all ebuilds:

IUSE="at-most-one-of ( sysv-utils systemd )"

RDEPEND=“
 	...
	sysv-utils? ( >=sys-apps/openrc-0.8.2-r1 )
	systemd? ( sys-apps/systemd )
 	...
 	“

The fact that there’s a hard dependency instead of a USE-conditional dependency is making Catalyst fail on systemd tarballs.
Comment 6 Mike Gilbert gentoo-dev 2019-09-27 15:55:15 UTC
Reviewing the livecd-tools ebuild, it only installs a few things:

1. Some OpenRC init scripts.

Clearly useless without OpenRC.

2. /usr/sbin/net-setup

Generates /etc/conf.d/net, so useless without OpenRC / netifrc.


3. /sbin/livecd-functions.sh

Sources /etc/init.d/functions.sh, so useless without OpenRC.


Given that livecd-tools relies heavily on OpenRC to do anything useful at all, I don't really understand your proposed changes.

Perhaps you would be better off changing catalyst to not install livecd-tools.
Comment 7 Kenneth G. Strawn 2019-09-27 16:38:36 UTC
Catalyst tries to install livecd-tools whether the user wants to or not — there’s no way to prevent it in the .spec files.
Comment 8 Kenneth G. Strawn 2019-09-27 19:10:49 UTC
For those on the Catalyst team ― the dependency of livecd-tools on OpenRC is creating a dependency blocking conflict when a systemd-based live media build is attempted. Remove livecd-tools from the dependency tree.
Comment 9 Kenneth G. Strawn 2019-09-27 19:54:50 UTC
Was able to find a temporary workaround: adding “!(app-misc/livecd-tools)” to the “livecd/packages” section of the .spec file seems to have done the trick.
Comment 10 Ben Kohler gentoo-dev 2019-09-28 14:34:46 UTC
FYI we're hard depending on livecd-tools because it provides the autologin on ttys.  We can try to work on making this also work on systemd, but it doesn't yet.

I'm curious how your systemd livecd login works without the livecd-tools "fixinittab" script.
Comment 11 Kenneth G. Strawn 2019-09-29 15:26:43 UTC
I’m trying to make a GNOME LiveDVD — that’s why it works, because I can just add the following lines to /etc/gdm/custom.conf to enable autologin (assuming I’ve provided a root password of course):

[daemon]
AutomaticLoginEnable=true
AutomaticLogin=root

For a KDE Plasma LiveDVD you can do something similar with /etc/sddm.conf:
[Autologin]
MinimumUid=0
User=root
Session=plasma.desktop

Granted, this won’t work if your LiveDVD is just a console, but it will definitely work if you’ve got a DE on it.
Comment 12 Kenneth G. Strawn 2019-09-29 15:32:16 UTC
Oops, made a mistake with the KDE Plasma example (/etc/sddm.conf):

[Users]
MinimumUid=0

[Autologin]
User=root
Session=plasma.desktop

And just for reiteration, here’s the GNOME example that I’m actually using (/etc/gdm/custom.conf):

[daemon]
AutomaticLoginEnable=true
AutomaticLogin=root
Comment 13 Kenneth G. Strawn 2019-09-29 15:34:07 UTC
In addition to modifying the GDM configuration files in my fsscript (using sed), I’m also pulling in app-admin/calamares in the “livecd/packages” section of my spec file — for improving the installation process.
Comment 14 Kenneth G. Strawn 2019-09-29 15:39:11 UTC
Here’s the lines of my fsscript that get root login enabled:

#Automatic Login
sed -i “3a AutomaticLoginEnable=true” /etc/gdm/custom.conf
sed -i “4a AutomaticLogin=root” /etc/gdm/custom.conf
Comment 15 Kenneth G. Strawn 2019-09-29 16:13:22 UTC
Created attachment 591248 [details]
Stage 1 specfile
Comment 16 Kenneth G. Strawn 2019-09-29 16:14:50 UTC
Created attachment 591250 [details]
Stage 2 specfile
Comment 17 Kenneth G. Strawn 2019-09-29 16:15:16 UTC
Created attachment 591252 [details]
fsscript
Comment 18 Kenneth G. Strawn 2019-09-29 16:16:11 UTC
Created attachment 591254 [details]
catalystrc

Added all files that my temporary workaround entails.
Comment 19 Kenneth G. Strawn 2019-09-29 16:24:02 UTC
Created attachment 591256 [details]
Updated fsscript
Comment 20 Kenneth G. Strawn 2019-09-29 16:50:40 UTC
So as you can see from all those spec files and/or scripts, I’m trying to make my live image as easy to install as possible — especially on UEFI-based systems, which are a pain to install GRUB on using the traditional console-based installation method.
Comment 21 Kenneth G. Strawn 2019-09-29 21:13:01 UTC
The only problem I’m having at this point is that emerging dev-lang/spidermonkey within the liveCD chroot is causing an OOM condition.
Comment 22 Mike Gilbert gentoo-dev 2019-09-29 21:39:02 UTC
The Arch wiki has some examples for tty auto-login using drop-in files to override ExecStart in getty@.service.

https://wiki.archlinux.org/index.php/Getty#Automatic_login_to_virtual_console
Comment 23 Kenneth G. Strawn 2019-09-30 13:28:50 UTC
Update: It seems that even blocking app-misc/livecd-tools didn't do the trick, so I had to run this instead:

sudo sed -i "45d" /usr/lib64/python3.6/site-packages/catalyst/targets/livecd_stage1.py

Once I did that, the build became completely flawless, up to and including the live login as I explained earlier with the display manager configuration file manipulation lines in my fsscript.
Comment 24 Kenneth G. Strawn 2019-09-30 13:50:39 UTC
To the Catalyst team: If removing livecd-tools from the hard dependencies is not possible, then at least make a systemd port of livecd-tools. That shouldn’t be hard to do at all.
Comment 25 Kenneth G. Strawn 2019-09-30 13:52:21 UTC
Also as Mike pointed out and per the Arch wiki, console autologin is definitely easy to accomplish via Getty config file editing.
Comment 26 Kenneth G. Strawn 2019-10-02 02:26:50 UTC
Created attachment 591540 [details]
Build automation utility

In case there is any lingering doubt that livecd-tools is not necessary on systemd targets, here's a utility I created to automate how I did it (which also includes bundled .spec files).
Comment 27 Kenneth G. Strawn 2019-10-04 13:23:33 UTC
Comment on attachment 591540 [details]
Build automation utility

Flagging this as obsolete; going to post an updated automation utility that fixes a few issues with this one after I post a screenshot of the LiveCD desktop.

Perfect timing: today is October 4, 2019, which means that Gentoo is 20 years old. Too bad there’s no official 20th anniversary edition out there
Comment 28 Kenneth G. Strawn 2019-10-04 14:32:30 UTC
Created attachment 591800 [details]
Wrapper script

Script I created to temporarily work around this bug until it is actually fixed
Comment 29 Kenneth G. Strawn 2019-10-04 14:35:41 UTC
Created attachment 591802 [details]
fsscript

The script that is loaded in the "livecd/fsscript" section of my specfile
Comment 30 Kenneth G. Strawn 2019-10-04 14:40:46 UTC
Created attachment 591804 [details]
fsscript
Comment 31 Kenneth G. Strawn 2019-10-04 14:46:46 UTC
Created attachment 591806 [details]
Script to resolve keyword issues because Catalyst fails to

Although unrelated to this in particular, the inability to specify "livecd/keywords:" in the .spec files is a problem. So I wrote a shell script to work around it.
Comment 32 Kenneth G. Strawn 2019-10-04 14:48:07 UTC
Created attachment 591808 [details]
livecd-stage1.spec
Comment 33 Kenneth G. Strawn 2019-10-04 14:49:24 UTC
Created attachment 591810 [details]
Kernel configuration
Comment 34 Kenneth G. Strawn 2019-10-04 14:51:12 UTC
Created attachment 591812 [details]
livecd-stage2.spec

Okay, I just reattached all the files necessary to actually make Catalyst work in its current state. Feel free to read through them, because the changes required are indeed extensive.
Comment 35 Ben Kohler gentoo-dev 2019-10-04 14:51:26 UTC
(In reply to Kenneth G. Strawn from comment #31)
> Created attachment 591806 [details]
> Script to resolve keyword issues because Catalyst fails to
> 
> Although unrelated to this in particular, the inability to specify
> "livecd/keywords:" in the .spec files is a problem. So I wrote a shell
> script to work around it.

You're supposed to use portage_confdir for that.

This bug report is quickly getting out of control with custom hacks attached, I think we need to focus on exactly what changes to livecd-tools & catalyst you propose, with real patches.
Comment 36 Mike Gilbert gentoo-dev 2019-10-04 14:54:27 UTC
Created attachment 591814 [details, diff]
Remove hard-coded livecd-tools
Comment 37 Mike Gilbert gentoo-dev 2019-10-04 14:59:17 UTC
Created attachment 591816 [details, diff]
Remove hard-coded livecd-tools

Added Closes: tag to commit message.
Comment 38 Ben Kohler gentoo-dev 2019-10-04 15:26:46 UTC
If we make this change, we're going to have to add this in specs for a lot of different builds & arches in releng.git.

This is doable but we need to make it a high priority or things will go haywire.
Comment 39 Mike Gilbert gentoo-dev 2019-10-04 15:35:16 UTC
Created attachment 591818 [details, diff]
Add app-misc/livecd-tools to all livecd-stage1 specs
Comment 40 Larry the Git Cow gentoo-dev 2019-10-11 15:59:14 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/releng.git/commit/?id=17e58eb7e5ca9b3a80a84aaeddb62233eb9a1623

commit 17e58eb7e5ca9b3a80a84aaeddb62233eb9a1623
Author:     Mike Gilbert <floppym@gentoo.org>
AuthorDate: 2019-10-04 15:33:11 +0000
Commit:     Ben Kohler <bkohler@gentoo.org>
CommitDate: 2019-10-11 15:56:53 +0000

    Add app-misc/livecd-tools to all livecd-stage1 specs
    
    Bug: https://bugs.gentoo.org/695616
    Signed-off-by: Mike Gilbert <floppym@gentoo.org>

 releases/weekly/specs/alpha/installcd-stage1.spec                | 1 +
 releases/weekly/specs/amd64/hardened/admincd-stage1-selinux.spec | 1 +
 releases/weekly/specs/amd64/hardened/admincd-stage1.spec         | 1 +
 releases/weekly/specs/amd64/installcd-stage1.spec                | 1 +
 releases/weekly/specs/amd64/livecd-stage1.spec                   | 1 +
 releases/weekly/specs/amd64/livedvd-stage1.spec                  | 1 +
 releases/weekly/specs/hppa/installcd-stage1.spec                 | 1 +
 releases/weekly/specs/ia64/installcd-stage1.spec                 | 1 +
 releases/weekly/specs/ppc/ppc32/installcd-stage1.spec            | 1 +
 releases/weekly/specs/sparc/sparc64/installcd-stage1.spec        | 1 +
 releases/weekly/specs/x86/hardened/admincd-stage1.spec           | 1 +
 releases/weekly/specs/x86/hardened/installcd-stage1.spec         | 1 +
 releases/weekly/specs/x86/i686/livecd-stage1.spec                | 1 +
 releases/weekly/specs/x86/i686/livedvd-stage1.spec               | 1 +
 releases/weekly/specs/x86/installcd-stage1.spec                  | 1 +
 15 files changed, 15 insertions(+)
Comment 41 Larry the Git Cow gentoo-dev 2019-10-11 16:02:34 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/proj/catalyst.git/commit/?id=e21da9dbef8bcf7795223727a1e71c3e83872436

commit e21da9dbef8bcf7795223727a1e71c3e83872436
Author:     Mike Gilbert <floppym@gentoo.org>
AuthorDate: 2019-10-04 14:49:54 +0000
Commit:     Ben Kohler <bkohler@gentoo.org>
CommitDate: 2019-10-11 16:02:06 +0000

    livecd_stage1.py: drop hard-coded app-misc/livecd-tools
    
    This belongs in the spec file if needed.
    
    Closes: https://bugs.gentoo.org/695616
    Signed-off-by: Mike Gilbert <floppym@gentoo.org>
    Signed-off-by: Ben Kohler <bkohler@gentoo.org>

 catalyst/targets/livecd_stage1.py | 1 -
 1 file changed, 1 deletion(-)