Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 482702 - Provide alternative stage3 tarballs using sys-apps/systemd
Summary: Provide alternative stage3 tarballs using sys-apps/systemd
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Release Media
Classification: Unclassified
Component: Stages (show other bugs)
Hardware: All Linux
: Normal normal with 1 vote (vote)
Assignee: Gentoo Release Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-08-27 18:27 UTC by Pacho Ramos
Modified: 2016-05-08 08:45 UTC (History)
10 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Pacho Ramos gentoo-dev 2013-08-27 18:27:15 UTC
Currently, people installing Gentoo at first time will need to use a stage3 using sys-fs/udev as device manager provider and, then, install sys-apps/systemd if they want to use that (either as device manager or to run all systemd parts). This is problematic as prevents us from setting "systemd" USE flag globally until sys-apps/systemd is not installing as, otherwise, people get a lot of blockers and circular dep issues that are hard to solve.

The suggestion would be to offer some stage3 tarballs using sys-apps/systemd as device manager provider instead of sys-fs/udev. Even, in the future, this could replace current stage3 as migration from sys-apps/systemd to sys-fs/udev is much easier (for people preferring it as device manager provider -> please remember that you can install sys-apps/systemd for device manager and keep using openrc), but last time we check looks like udev team wasn't ok with this approach. Then, the "provide as an alternative" would be enough :)

Thanks a lot
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-08-27 18:33:02 UTC
Just to be clear here, it's not really udev team who's blocking this but the so-called community.

As far as I'm aware the only 'non-fixable' technical argument is that systemd has additional dep on dbus. Other stuff can either be changed or is moot (like the general idea of hating having systemd parts around).
Comment 2 Samuli Suominen (RETIRED) gentoo-dev 2013-08-28 10:05:02 UTC
sys-apps/systemd doesn't work as sys-fs/udev alternative as-is, the dbus dependency is one thing, but there are also:

- intltool/gettext conditionalizing
- -accept4.patch Linux 2.6.32+openrc+udev support
- the manpage building dependencies, docbook, libxslt, and co
- wrong installation path, should be / instead of /usr
- restoration of libudev.a building, restoration of USE="static-libs"
- restoration of <gcc-4.6 support, but I guess we could let this go now
- restoration of sys-libs/uclibc support by secure_getenv() workaround

plus number of other technical ebuild issues that are more or less obvious when you go through sys-fs/udev's ebuild line-by-line

mostly everything sys-fs/udev does, it does for a good reason, and migration from sys-fs/udev -> sys-apps/systemd regarding OpenRC users shouldn't be introducing any regressions for them
ie. any possible migration would need to be completely transparent, down to using profiles/updates/ for it
i'd be open to ditching sys-fs/udev, but i won't even consider it before systemd's ebuild is in order...

the only 'politics', or 'so-called community' issues in my eyes are really 'lack of trust from udev maintainers towards systemd maintainers for not introducing any of these technical regressions, in favour of their design model' :/
Comment 3 Anthony Basile gentoo-dev 2013-08-28 10:22:55 UTC
(In reply to Samuli Suominen from comment #2)
> 
> the only 'politics', or 'so-called community' issues in my eyes are really
> 'lack of trust from udev maintainers towards systemd maintainers for not
> introducing any of these technical regressions, in favour of their design
> model' :/

I wouldn't say lack of trust so much as we have different priorities and so stages which require systemd -> udev migration would switch the maintenance burdon to the udev users and developers.

If there really is no easy migration path, then maybe a systemd stage in addition to the udev stage would be useful.  I would recommend only amd64 and x86.

Note that the releng team is understaffed.
Comment 4 Pacho Ramos gentoo-dev 2013-08-28 10:38:16 UTC
(In reply to Samuli Suominen from comment #2)
> sys-apps/systemd doesn't work as sys-fs/udev alternative as-is, the dbus
> dependency is one thing, but there are also:
> 

Well, this was the main reason for suggesting to provide both alternatives in the short term, but I also suggested the other one as a long term solution if we finally reach a way to merge some of the efforts.

> - intltool/gettext conditionalizing

It's the first time I am warned about this being an issue, will need to review it :/ Are you referring to this?
        # Remove requirements for gettext and intltool wrt bug #443028
        if ! has_version dev-util/intltool && ! [[ ${PV} = 9999* ]]; then
                sed -i \
                        -e '/INTLTOOL_APPLIED_VERSION=/s:=.*:=0.40.0:' \
                        -e '/XML::Parser perl module is required for intltool/s|^|:|' \
                        configure || die
                eval export INTLTOOL_{EXTRACT,MERGE,UPDATE}=/bin/true
                eval export {MSG{FMT,MERGE},XGETTEXT}=/bin/true
        fi


Reading the bug report, looks like this causes problems with ulibc and, on systemd side, since we have a policykit USE flag, maybe we could apply it also there, mgorny?

> - -accept4.patch Linux 2.6.32+openrc+udev support

How is this affecting to stage3 users? 2.6.32 gentoo-sources are not even available and, then, people using stage3 to install Gentoo won't likely install that unmaintained kernel. Also, why is so important to keep compat with 2.6.32 over 2.6.39?

> - the manpage building dependencies, docbook, libxslt, and co

How are you skipping that deps in sys-fs/udev currently? (I failed to see the "tricks" as man pages installation is a bit "complicated" in that ebuild).

> - wrong installation path, should be / instead of /usr

As talked with mgorny, looks like he would be willing to change that if we are going to allow systemd ebuild supply udev also (it's a problem like "snake eating his own tail" (not sure if that is the way it's said in English))

> - restoration of libudev.a building, restoration of USE="static-libs"

This is probably the worst part from my point of view as I also doubt we should keep providing this as upstream is not maintaining and supporting it at all :|

> - restoration of <gcc-4.6 support, but I guess we could let this go now
> - restoration of sys-libs/uclibc support by secure_getenv() workaround
> 

No idea about this :S, maybe other systemd team members can help
Comment 5 Pacho Ramos gentoo-dev 2013-08-28 10:40:14 UTC
(In reply to Anthony Basile from comment #3)
> If there really is no easy migration path, then maybe a systemd stage in
> addition to the udev stage would be useful.  I would recommend only amd64
> and x86.
> 
> Note that the releng team is understaffed.

Would be nice, but I would also try to supply also an arm one if possible (if it's too hard, no problem of course). Well, systemd is also keyworded on ppc* but I doubt many users will benefit from it as that arch is deprecated, but arm should keep growing :)

Thanks
Comment 6 Anthony Basile gentoo-dev 2013-08-28 11:08:02 UTC
(In reply to Pacho Ramos from comment #5)
> (In reply to Anthony Basile from comment #3)
> > If there really is no easy migration path, then maybe a systemd stage in
> > addition to the udev stage would be useful.  I would recommend only amd64
> > and x86.
> > 
> > Note that the releng team is understaffed.
> 
> Would be nice, but I would also try to supply also an arm one if possible
> (if it's too hard, no problem of course). Well, systemd is also keyworded on
> ppc* but I doubt many users will benefit from it as that arch is deprecated,
> but arm should keep growing :)
> 
> Thanks

Yeah, arm may even be more important than x86.
Comment 7 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2013-08-28 11:37:51 UTC
Just a warning, if this turns into a udev/systemd "mud-slinging" bug, I'll drop release from it. Also, before anyone gets "carried away", I'm not ready to replace udev with systemd anywhere in the near future.
If anyone wants to have stages with systemd, someone will have to step forward and work on this. I suggest such stages should initially be put on the /experimental tree.
Comment 8 Samuli Suominen (RETIRED) gentoo-dev 2013-08-28 12:41:03 UTC
(In reply to Jorge Manuel B. S. Vicetto from comment #7)
> Just a warning, if this turns into a udev/systemd "mud-slinging" bug, I'll
> drop release from it.

I think everyone has been civil and only technical issues have been talked about, meant as a reply to second half of Comment #0 :)

> Also, before anyone gets "carried away", I'm not ready
> to replace udev with systemd anywhere in the near future.

You propably didn't understand the intention, the ebuild category or package name doesn't really matter if the ran ebuild code is identical with defined USE flags, I mean, USE="-systemd" in sys-apps/systemd would equal ebuild of sys-fs/udev

Users would see portage automatically moving eg. =sys-fs/udev-206-rX to =sys-apps/systemd-206-rX and have USE="-systemd" and the installed files are more or less same.

(Likewise it would be possible to add USE="systemd" to sys-fs/udev and install the rest from there. Just saying the same really doesn't matter, except that the upstream tarball/package name is often also the ebuild package name.)

Surely you have to agree if everything gets technically sorted out that upgrading from =sys-fs/udev-206-rX to =sys-apps/systemd-206-rX[-systemd] gives you the same package, just something minor as ebuild name, doesn't warrant having duplicate ebuild in gentoo-x86.

> If anyone wants to have stages with systemd, someone will have to step
> forward and work on this. I suggest such stages should initially be put on
> the /experimental tree.

yeah
Comment 9 Alexander Tsoy 2013-08-29 16:03:50 UTC
Automated build of such stage3 with a catalyst is impossible until bug 481920 and bug 454462 are resolved. Moreover systemd profile should pull sys-apps/systemd in a @system set.

But it is possible to build stage4 target. I already got it to work with some hacks in stage4/fsscript to bypass circular deps between dbus and systemd.
Comment 10 Pacho Ramos gentoo-dev 2013-08-29 17:06:51 UTC
(In reply to Alexander Tsoy from comment #9)
> Automated build of such stage3 with a catalyst is impossible until bug
> 481920 and bug 454462 are resolved. Moreover systemd profile should pull
> sys-apps/systemd in a @system set.
> 

It's a bit like a circular problem: we cannot enable systemd USE flag system wide if sys-apps/systemd is not already installed, for that stage3 with that would be great (even being built with USE "-systemd")
Comment 11 Alexander Tsoy 2013-08-30 01:01:15 UTC
(In reply to Pacho Ramos from comment #10)
> (In reply to Alexander Tsoy from comment #9)
> > Automated build of such stage3 with a catalyst is impossible until bug
> > 481920 and bug 454462 are resolved. Moreover systemd profile should pull
> > sys-apps/systemd in a @system set.
> > 
> 
> It's a bit like a circular problem: we cannot enable systemd USE flag system
> wide if sys-apps/systemd is not already installed, for that stage3 with that
> would be great (even being built with USE "-systemd")

By the way I have a script that simplifies build of stage3/stage4 (because using catalyst directly is a hell) and makes it possible to do this by a cron job. I'm using it for a long time to build openvz templates. Adding support for stages with systemd took me about 5 minutes (it is actually a stage4 in the terms of catalyst, but this doesn't matter). I can share this script and all necessary files with you. Contact me by email if you interested.
Comment 12 Alexander Tsoy 2013-09-01 21:50:59 UTC
(In reply to Alexander Tsoy from comment #9)
> Automated build of such stage3 with a catalyst is impossible until bug
> 481920 and bug 454462 are resolved. Moreover systemd profile should pull
> sys-apps/systemd in a @system set.

Forget this comment. :) I succesfully built stage3 with systemd. The only difference from the "normal" stage3 build process: stage1 was built with an option "portage_confdir" pointing to the directory containing 2 files:

# cat package.mask 
sys-fs/udev
# cat package.use 
sys-apps/hwids udev

http://tsoy.me/pub/gentoo-stage3/
Comment 13 Pacho Ramos gentoo-dev 2013-09-11 08:45:48 UTC
(In reply to Alexander Tsoy from comment #12)
> (In reply to Alexander Tsoy from comment #9)
> > Automated build of such stage3 with a catalyst is impossible until bug
> > 481920 and bug 454462 are resolved. Moreover systemd profile should pull
> > sys-apps/systemd in a @system set.
> 
> Forget this comment. :) I succesfully built stage3 with systemd. The only
> difference from the "normal" stage3 build process: stage1 was built with an
> option "portage_confdir" pointing to the directory containing 2 files:
> 
> # cat package.mask 
> sys-fs/udev
> # cat package.use 
> sys-apps/hwids udev
> 
> http://tsoy.me/pub/gentoo-stage3/

@releng, I am unsure what would be required to get them merged "officially". Are stages build by ourselves and, later, we upload it to some place or, once an updated "script" is provided, releng team has some automated way to generate stages periodically? Thanks for the info
Comment 14 Pacho Ramos gentoo-dev 2013-09-14 20:57:38 UTC
(In reply to Pacho Ramos from comment #0)
> Currently, people installing Gentoo at first time will need to use a stage3
> using sys-fs/udev as device manager provider and, then, install
> sys-apps/systemd if they want to use that (either as device manager or to
> run all systemd parts). This is problematic as prevents us from setting
> "systemd" USE flag globally until sys-apps/systemd is not installing as,
> otherwise, people get a lot of blockers and circular dep issues that are
> hard to solve.
> 

A much simpler solution would be to simply include sys-apps/dbus in stage3, what about that release team?
Comment 15 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2013-09-15 00:07:42 UTC
(In reply to Pacho Ramos from comment #14)
> (In reply to Pacho Ramos from comment #0)
> > Currently, people installing Gentoo at first time will need to use a stage3
> > using sys-fs/udev as device manager provider and, then, install
> > sys-apps/systemd if they want to use that (either as device manager or to
> > run all systemd parts). This is problematic as prevents us from setting
> > "systemd" USE flag globally until sys-apps/systemd is not installing as,
> > otherwise, people get a lot of blockers and circular dep issues that are
> > hard to solve.
> > 
> 
> A much simpler solution would be to simply include sys-apps/dbus in stage3,
> what about that release team?

The build of stage3 is done by emerging system. If you want to get dbus in the stages, you'll have to get dbus on system.
The alternative is to get bug 393445 fixed. In that case, we'll have to discuss if dbus should be on all stage3 tarballs.
Comment 16 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2013-09-15 00:12:42 UTC
(In reply to Pacho Ramos from comment #13)
> @releng, I am unsure what would be required to get them merged "officially".
> Are stages build by ourselves and, later, we upload it to some place or,
> once an updated "script" is provided, releng team has some automated way to
> generate stages periodically? Thanks for the info

Automated stages are build using the specs in the release overlay[1]. If you have any stages with systemd, they can be added to the experimental tree[2].
Finally, as I've stated before, I'm not ready to replace udev with systemd anywhere near in the future on stages, so there's no point in trying to push that.

 [1] - http://git.overlays.gentoo.org/gitweb/?p=proj/releng.git
 [2] - http://distfiles.gentoo.org/experimental/
Comment 17 Pacho Ramos gentoo-dev 2013-09-15 08:19:10 UTC
bug 393445 looks more reasonable, but what is blocking it? (I don't see many comments there, neither blocks to the bug)
Comment 18 Anthony Basile gentoo-dev 2013-09-15 11:16:37 UTC
(In reply to Jorge Manuel B. S. Vicetto from comment #16)
> (In reply to Pacho Ramos from comment #13)
> > @releng, I am unsure what would be required to get them merged "officially".
> > Are stages build by ourselves and, later, we upload it to some place or,
> > once an updated "script" is provided, releng team has some automated way to
> > generate stages periodically? Thanks for the info
> 
> Automated stages are build using the specs in the release overlay[1]. If you
> have any stages with systemd, they can be added to the experimental tree[2].
> Finally, as I've stated before, I'm not ready to replace udev with systemd
> anywhere near in the future on stages, so there's no point in trying to push
> that.
> 
>  [1] - http://git.overlays.gentoo.org/gitweb/?p=proj/releng.git
>  [2] - http://distfiles.gentoo.org/experimental/

I would prefer to see systemd stage3's in experimental rather than dbus added to all stage3's that we push out in release.  Those stage3's should remain as neutral as possible as they are used as seeds to building all sorts of systems; in particular, systems that have no need for dbus.
Comment 19 David Heidelberg (okias) 2014-01-28 21:06:10 UTC
Reminder. Btw., systemd-9999 +kdbus is actually already able to replace dbus, if it's matter. Of course, depends on kdbus kernel module (compiled in kernel).
Comment 20 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2014-02-07 12:36:01 UTC
(In reply to David Heidelberger (okias) from comment #19)
> Reminder. Btw., systemd-9999 +kdbus is actually already able to replace
> dbus, if it's matter. Of course, depends on kdbus kernel module (compiled in
> kernel).

@systemd:

What are the plans for the near future? Will you need dbus for quite sometime, or do you think a systemd version requiring just kdbus should be around soon?

I'm asking as I believe systemd + kdbus will be a lot easier for users as there won't be any circular dependencies.
Comment 21 David Heidelberg (okias) 2014-03-07 00:24:17 UTC
(In reply to Jorge Manuel B. S. Vicetto from comment #20)
> (In reply to David Heidelberger (okias) from comment #19)
> > Reminder. Btw., systemd-9999 +kdbus is actually already able to replace
> > dbus, if it's matter. Of course, depends on kdbus kernel module (compiled in
> > kernel).
> 
> @systemd:
> 
> What are the plans for the near future? Will you need dbus for quite
> sometime, or do you think a systemd version requiring just kdbus should be
> around soon?

Well, kdbus is nice (as I having it on one machine perfectly running), but we can't ensure that kernel will be compiled with it. In that case there should be working dbus, to prevent possible un-bootable state.

> 
> I'm asking as I believe systemd + kdbus will be a lot easier for users as
> there won't be any circular dependencies.

Yes, but it will take probably some time. We could require kdbus, but it's not even proposed to merge to Linux kernel, so I not expecting to hit it before 3.16.
Comment 22 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2014-03-07 04:00:59 UTC
(In reply to David Heidelberger (okias) from comment #21)
> (In reply to Jorge Manuel B. S. Vicetto from comment #20)
> > What are the plans for the near future? Will you need dbus for quite
> > sometime, or do you think a systemd version requiring just kdbus should be
> > around soon?
> 
> Well, kdbus is nice (as I having it on one machine perfectly running), but
> we can't ensure that kernel will be compiled with it. In that case there
> should be working dbus, to prevent possible un-bootable state.

OK, I understand that, but we're talking about stage3 here. So if kdbus is / becomes an option, perhaps we can move the dbus dependency out of system and thus avoid the circular dependencies.
Then dbus only needs to be pulled in when building the CDs.
Comment 23 Samuli Suominen (RETIRED) gentoo-dev 2014-03-07 05:49:50 UTC
(In reply to Jorge Manuel B. S. Vicetto from comment #22)
> (In reply to David Heidelberger (okias) from comment #21)
> > (In reply to Jorge Manuel B. S. Vicetto from comment #20)
> > > What are the plans for the near future? Will you need dbus for quite
> > > sometime, or do you think a systemd version requiring just kdbus should be
> > > around soon?
> > 
> > Well, kdbus is nice (as I having it on one machine perfectly running), but
> > we can't ensure that kernel will be compiled with it. In that case there
> > should be working dbus, to prevent possible un-bootable state.
> 
> OK, I understand that, but we're talking about stage3 here. So if kdbus is /
> becomes an option, perhaps we can move the dbus dependency out of system and
> thus avoid the circular dependencies.

What circular dependencies? Newer systemd ebuilds only have sys-apps/dbus in PDEPEND, thus, first sys-apps/systemd gets built, then sys-apps/dbus[systemd] gets built.
Older versions of systemd ebuilds have it in "COMMON_DEPEND" thing, but not newer, so stabilizing eg. 210, would help.
Comment 24 Anthony Basile gentoo-dev 2016-05-07 15:08:41 UTC
Okay I think this bug is done, so I'm going to close it.  I'm building and will continue to build amd64 and x86 systemd stages.

@Pacho.  This was your bug, so if you're not satisfied for some reason, or there's still some issue left unaddressed, please re-open it.

Thanks guys!
Comment 25 Pacho Ramos gentoo-dev 2016-05-08 08:45:05 UTC
Thanks to you for the work! :)