Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 437570 - standalone udev
Summary: standalone udev
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: Normal enhancement with 9 votes (vote)
Assignee: eudev team
URL:
Whiteboard:
Keywords:
Depends on: 444398
Blocks:
  Show dependency tree
 
Reported: 2012-10-08 04:27 UTC by grey dot
Modified: 2013-01-28 14:40 UTC (History)
27 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
streq() missing (udev-9999-streq.patch,328 bytes, patch)
2012-10-10 18:48 UTC, Jimmy.Jazz
Details | Diff
udev-9999.ebuild altered (udev-9999.ebuild.diff,3.14 KB, patch)
2012-10-10 18:57 UTC, Jimmy.Jazz
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description grey dot 2012-10-08 04:27:05 UTC
The forked udev is stable and tested enough to be used on a regular basis. What would it require for us to include it into the main portage tree?

Ebuilds are available in the udev overlay.
Comment 1 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2012-10-08 06:07:38 UTC
Can you link to it at least?
Comment 2 Andreas Sturmlechner gentoo-dev 2012-10-08 06:43:46 UTC
If I might: https://bitbucket.org/braindamaged/udev
Comment 3 grey dot 2012-10-08 06:50:29 UTC
(In reply to comment #1)
> Can you link to it at least?

And this is the overlay:
https://bitbucket.org/braindamaged/udev-overlay.git
Comment 4 Samuli Suominen gentoo-dev 2012-10-08 06:55:37 UTC
Nothing for udev-bugs@ to do here since we maintain upstream. We can talk once you have some credible kernel hackers behind the idea of forking.
So far everything has been merely cosmetics and not much else than idiotic sep. /usr layout from 90's got obsoleted.
If you meant the later firmware loading bug, then file a real bug for it.
Comment 5 grey dot 2012-10-08 07:13:55 UTC
(In reply to comment #4)
> Nothing for udev-bugs@ to do here since we maintain upstream. We can talk
> once you have some credible kernel hackers behind the idea of forking.
> So far everything has been merely cosmetics and not much else than idiotic
> sep. /usr layout from 90's got obsoleted.
> If you meant the later firmware loading bug, then file a real bug for it.

I remember you wrote the same in email and were told your arguments are invalid. I'm not proposing removing systemd-udev from portage and replacing it with this. I merely suggest including it into a main portage tree (under some other name perhaps) and adding as a dependency to virtual/dev-manager.
Comment 6 Steve L 2012-10-08 08:10:37 UTC
(In reply to comment #4)
> Nothing for udev-bugs@ to do here since we maintain upstream.

You maintain the gentoo ebuild of the upstream package, yes. That doesn't
qualify you to veto an ebuild for an alternative implementation.

You were asked about how this should be integrated into Gentoo, so as not
to mess up end-users' machines, with conflicts either in the portage tree
or the end binaries.

> We can talk once you have some credible kernel hackers behind
> the idea of forking.

What like Linus Torvalds?
 "What kind of insane udev maintainership do we have?
  And can we fix it?" [1]
Alan Cox?
 "Just fix udev, and if you can't fix it someone please just 
 fork the last working one." [2]
Al Viro?
 "I'd still prefer to see udev forcibly taken over." [3]

[1] http://marc.info/?l=linux-kernel&m=134919561607889&w=2
[2] http://marc.info/?l=linux-kernel&m=134930049122707&w=2
[3] http://marc.info/?l=linux-kernel&m=134929242315036&w=2

> So far everything has been merely cosmetics and not much else than idiotic
> sep. /usr layout from 90's got obsoleted.

Well, translating that into English, you appear to be saying that because you
don't see the utility in a separate /usr layout, you therefore believe
it to be obsolete.

Your opinion of what users choose to do with their machines, while cogent
to what you advise, is not relevant to what choices should be provided. You
are a volunteer for the supplier, not the customer.

The simple truth is that people complaining about udev's incompatible changes
were told "If you want it different, fork the code. Until someone puts that
work in, nothing else can be provided."

They've put the work in. The forum thread [4] shows that it's working for
quite a few people, and there is clearly user-demand for such a solution.

All they're asking is: how do we make this work nicely in Gentoo.

Frankly, it's churlish simply to dismiss the whole project, and their hard
work, and the antithesis of how Gentoo usually deals with its users, who
are its potential developer pool.

[4] http://forums.gentoo.org/viewtopic-t-934678.html
Comment 7 William Hubbs gentoo-dev 2012-10-08 15:02:48 UTC
Here is my concern about this project (taken from the overview):

"
This is the systemd-ectomized version of udev. Most features from the upstream
are backported, though we do not guarantee 100% API compatibility.
"

In other words, this is not a drop-in replacement for udev.
Comment 8 grey dot 2012-10-08 15:09:03 UTC
(In reply to comment #7)
> Here is my concern about this project (taken from the overview):
> 
> "
> This is the systemd-ectomized version of udev. Most features from the
> upstream
> are backported, though we do not guarantee 100% API compatibility.
> "
> 
> In other words, this is not a drop-in replacement for udev.

Forget these other words. As for now, this project is binary compatible with systemd-udev. And, again, I do NOT propose replacing systemd-udev with this, I just ask what it will take to include this into the main portage tree and to dev-manager dependency list. Period.
Comment 9 Khayyam 2012-10-08 22:22:07 UTC
(In reply to comment #4)
> Nothing for udev-bugs@ to do here since we maintain upstream.
Samuli ... and you will no doubt be aware that upstream has made it quite clear that "udev on non-systemd systems is in our eyes a dead end, in case you haven't noticed it yet. I am looking forward to the day when we can drop that support entirely." [1] This will mean that in order to stick with upstream Gentoo will be required to switch to systemd, so having a replacement for udev, something that supports openrc *and* can fill the role currently occupied by systemd-udev may be necessary, that is, unless you think that Gentoo should drop openrc and follow the path being layed out by the systemd developers. So, it would seem to me that an alternate to systemd-udev is required here so that the choice of init can be maintained.

As you have pointed out in the past [2] virtual/device-mapper contains various replacements for systemd-udev, and that seems to be all that is being asked: what steps need to be taken for the fork to find its way into the virtual. Admittedly, the initial bug should have had some more thought behind it, and obviously the fork being still named "udev" is a source of possible confusion, but I think grey_dot intended it simply as preliminary equiry, and so rather than wash your hands of it as "[n]othing for udev-bugs@", you might have been a little more helpfull, particularly as you'd previously called for a "reality check." and stated that "If people don't write code to the contrary, their opinion is also meaningless. They will just have to adapt, even if they don't like it." [3] So, the code has been forked, and though no doubt there are outstanding issues (such as the name change, fitness for purpose, status as a "drop in replacement", issues regarding seperate /usr and effected dependencies, etc) that action has been taken, and so its beyond the stage of offering "opinions". What is requrired now is some communication between the various parties as to how to proceed, what issues are involved, and how to move ahead.
    
> We can talk once you have some credible kernel hackers behind the idea of forking.
I think steveL has provided such a "credible" list, but its somewhat besides the point, it seems that such a request is just posturing on your part.  

> So far everything has been merely cosmetics and not much else than idiotic
> sep. /usr layout from 90's got obsoleted.
grey_dot and concus can provide more information on how "cosmetic" the changes are ... or not, but I'm not sure why your so willing to dismiss it. As of the council meeting of April 2012 this idiocy is a supported configuration. [4] But even if "idiocy" there are other issues involved, as has been stated above with regard to "udev on non-systemd systems".

best ... khay

[1] http://lists.freedesktop.org/archives/systemd-devel/2012-August/006066.html
[2] http://forums.gentoo.org/viewtopic-p-6987772.html#6987772
[3] http://forums.gentoo.org/viewtopic-p-6987380.html#6987380
[4] http://www.gentoo.org/proj/en/council/meeting-logs/20120403-summary.txt
Comment 10 William Hubbs gentoo-dev 2012-10-09 15:48:12 UTC
I think we should hold off on putting this in the tree until the
upstream debate on lkml is over.
There is already a commit in the kernel which means that the kernel will
load firmware directly from the filesystem; this will hit in 3.7.

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commit;h=abb139e75c2cdbb955e840d6331cb5863e409d0e
Comment 11 William Hubbs gentoo-dev 2012-10-10 15:54:21 UTC
I checked LKML just now and they are saying that this may be brought
to the stable kernels upstream as well.
Comment 12 grey dot 2012-10-10 17:07:37 UTC
(In reply to comment #11)
> I checked LKML just now and they are saying that this may be brought
> to the stable kernels upstream as well.

Still, I don't quite understand what does it have to do with this udev fork. The purpose of this fork is to get rid of redundant dependencies such as dbus or systemd, and not to fix that bug. Though, we can try to fix that.
Comment 13 Jimmy.Jazz 2012-10-10 18:48:23 UTC
Created attachment 326214 [details, diff]
streq() missing

probably an oversight
Comment 14 Jimmy.Jazz 2012-10-10 18:57:13 UTC
Created attachment 326216 [details, diff]
udev-9999.ebuild altered

I liked the rootfs-install idea and let people install it in / or /usr.
The udev-9999 ebuild fork I'm using in attachment.
Comment 15 Lars Wendler (Polynomial-C) gentoo-dev 2012-10-10 20:11:57 UTC
(In reply to comment #14)
> Created attachment 326216 [details, diff] [details, diff]
> udev-9999.ebuild altered
> 
> I liked the rootfs-install idea and let people install it in / or /usr.
> The udev-9999 ebuild fork I'm using in attachment.

Don't worry. I'm planning to add braindead-udev to the tree (as soon as I have my dev machine back online) and let it install everything in /(s)bin and /lib where it really belongs to.
That should also keep everyone with a separate /usr partition and no initrd away from all the hassle our current unstable udev ebuilds impose on them for no good reason.
Comment 16 Lars Wendler (Polynomial-C) gentoo-dev 2012-10-10 20:12:39 UTC
er... I mean braindamaged-udev of course...
Comment 17 Khayyam 2012-10-11 04:49:07 UTC
(In reply to comment #15)
 
> Don't worry. I'm planning to add braindead-udev to the tree (as soon as I
> have my dev machine back online) and let it install everything in /(s)bin
> and /lib where it really belongs to.
> That should also keep everyone with a separate /usr partition and no initrd
> away from all the hassle our current unstable udev ebuilds impose on them
> for no good reason.

Lars .. this is good to hear. However, some things that perhaps should be discussed and thought through before this happens.

1). Should a name change be considered (grey_dot?). I'd suggested 'nudev', but I'll leave that to the fork mantainters to decide. It may be a good idea to do this prior to the inclusion in portage, as it would then be clearly differenciated from 'systemd-udev' (sys-fs/udev). Symlinks /sbin/udevadm -> nudevadm, etc, could be provided for compatability.

2). For separate /usr packages that udev links to such as kmod, which in turn links to xz-utils (and which the current gentoo package installs to /usr) will also need to be in / eg:

# ldd /sbin/udevd
	linux-gate.so.1 (0xb77db000)
	libblkid.so.1 => /lib/libblkid.so.1 (0xb77a2000)
	libkmod.so.2 => /lib/libkmod.so.2 (0xb778d000)
	librt.so.1 => /lib/librt.so.1 (0xb7784000)
	libc.so.6 => /lib/libc.so.6 (0xb75fb000)
	libuuid.so.1 => /lib/libuuid.so.1 (0xb75f4000)
	liblzma.so.5 => /lib/liblzma.so.5 (0xb75d1000)
	libz.so.1 => /lib/libz.so.1 (0xb75bc000)
	libpthread.so.0 => /lib/libpthread.so.0 (0xb75a2000)
	/lib/ld-linux.so.2 (0xb77dc000)

I can forsee this getting complicated as udev has a number of useflags/dependencies that will be in /usr, which in turn will mean wider package changes. This is a much bigger problem than the package inclusion, and so some thought needs to be given as to how this can be handled (if indeed it can be in the long term).

3). With the fork parting ways from systemd and to a lesser degree dbus, and systemd-udev tightening their intergration, will packages that use one or other necessarily work with either systemd-udev or the fork. Or, to put it another way, if having one as virtual/device-mapper will applications that USE="dbus" continue to function without that intergration? I don't use dbus, so I don't paricularly care if it breaks in that regard, but this is another area in which I can forsee possible issues.     

best ... khay
Comment 18 grey dot 2012-10-11 05:33:27 UTC
(In reply to comment #13)
> Created attachment 326214 [details, diff] [details, diff]
> streq() missing
> 
> probably an oversight

Thanks, already fixed that.

(In reply to comment #17)
> Should a name change be considered (grey_dot?). I'd suggested 'nudev', but I'll leave that to the fork mantainters to decide. 

I think sudev (s for standalone) will be alright. Or should we run a poll?

>I can forsee this getting complicated as udev has a number of useflags/dependencies that will be in /usr, which in turn will mean wider package changes. This is a much bigger problem than the package inclusion, and so some thought needs to be given as to how this can be handled (if indeed it can be in the long term).

At the moment the only dependencies triggers by use flags and located in /usr are those of libgudev, which isn't required during boot stage.

~/devel/udev/udev> ldd .libs/libgudev-1.0.so.0.1.2|grep /usr
	libgobject-2.0.so.0 => /usr/lib64/libgobject-2.0.so.0 (0x00007f3b5cb18000)
	libglib-2.0.so.0 => /usr/lib64/libglib-2.0.so.0 (0x00007f3b5c7f0000)
	libffi.so.6 => /usr/lib64/libffi.so.6 (0x00007f3b5c030000)

So, the only packages required to move to / are kmod and xz-utils. I hope no extra dependencies will be added.

>3). With the fork parting ways from systemd and to a lesser degree dbus, and systemd-udev tightening their intergration,

Well, I wouldn't say somebody is tightening something. The integration code is at most additional udev rules to allow systemd to be 'aware' of ongoing things, and that doesn't affect udev in any way. The code itself isn't affected

I suppose the same is for dbus (though I don't use it either).
Comment 19 Andreas Sturmlechner gentoo-dev 2012-10-11 06:40:59 UTC
(In reply to comment #12)
> (In reply to comment #11)
> > I checked LKML just now and they are saying that this may be brought
> > to the stable kernels upstream as well.
> 
> Still, I don't quite understand what does it have to do with this udev fork.

You could read it as a first sign of a change of mind from the kernel devs, whatever that might lead to. Al Viro taking over could gain wide-spread adoption in no time.
Comment 20 Lars Wendler (Polynomial-C) gentoo-dev 2012-10-11 10:09:09 UTC
Well the sys-apps/kmod stuff being in /usr might become a problem as that package gets maintained by the same narrow-minded developers who also maintain sys-fs/udev. I worked around that problem by having modified kmod ebuilds in my private overlay (poly-c) but of course that's no solution once we want to have braindamaged-udev in tree. One solution to this problem might be another package (e.g. liberated-kmod) that simply uses the kmod sources and installs all necessary files into / as well.
To be honest it's about time to start working against this asinine / --> /usr merge that the systemd/udev/kmod fanatics started to impose on us. 
That's my main motivation behind getting braindamaged-udev into the tree.
Comment 21 grey dot 2012-10-11 10:15:32 UTC
(In reply to comment #20)
> Well the sys-apps/kmod stuff being in /usr might become a problem as that
> package gets maintained by the same narrow-minded developers who also
> maintain sys-fs/udev. I worked around that problem by having modified kmod
> ebuilds in my private overlay (poly-c) but of course that's no solution once
> we want to have braindamaged-udev in tree. One solution to this problem
> might be another package (e.g. liberated-kmod) that simply uses the kmod
> sources and installs all necessary files into / as well.

Udev overlay also provides ebuilds to install kmod and its dependencies to / instead of /usr.

> To be honest it's about time to start working against this asinine / -->
> /usr merge that the systemd/udev/kmod fanatics started to impose on us. 
> That's my main motivation behind getting braindamaged-udev into the tree.

I'm having the same idea. But how?
Comment 22 Steve L 2012-10-11 10:36:09 UTC
(In reply to comment #18)
> I think sudev (s for standalone) will be alright. Or should we run a poll?
Not getting into bikeshedding, but as soon as I read 'sudev' I thought it was
something along the lines of su/sudo, to allow a user in a group to change perms
on a device node. A poll on the forums sounds right, so we don't bikeshed here.

> The integration code is at most additional udev rules to allow systemd to be
> 'aware' of ongoing things, and that doesn't affect udev in any way. The code
> itself isn't affected

That's interesting. It would be useful, in that case, to provide the same systemd rules under a USE flag, so that the fork can support all current use-cases: this will make it more viable as a replacement in the longer-term.
(If you're already doing that, sorry: it's just that what's needed is not a "systemd-ectomised" version, but rather a version that doesn't dictate any particular userland setup.)

(In reply to comment #19)
You've missed out the previous part of the conversation which was all about the
firmware bug, and ignored the wider issues. In that context, what greydot said
made perfect sense. Selectively quoting after the event to make a further point
doesn't change that.

With regard to your further point, the kernel hackers are basically pleading for
someone else to pitch in. Even if Al Viro, or anyone else, finds the time to
maintain yet another (major) project, they will have to roll-back recent udev
changes, and go back to a working version before 174 (personally I think 171-r6
in stable is pretty good) and then roll-forward stuff that isn't insane. IOW
they'll have to do the exact same work that has already been done.

So all this does is give everyone a head-start, and something to collaborate
around that works on real end-user machines.

As already stated, there have been minimal changes to the codebase, and everyone
would dearly love to keep it like that, using configure switches for /lib
instead of /usr/lib and so on.

Gentoo is the perfect place to do this, since it is from-source, the users are
very knowledgeable and provide excellent QA feedback, and the developers know
build-systems much better than most. If you want precedent, look at hardened
that was dead a few years ago, til a couple of users took it upon themselves to
put in the hard work, which fed into gcc-4 on other distros.
Comment 23 William Hubbs gentoo-dev 2012-10-15 16:57:27 UTC
(In reply to comment #20)
> Well the sys-apps/kmod stuff being in /usr might become a problem as that
> package gets maintained by the same narrow-minded developers who also
> maintain sys-fs/udev. I worked around that problem by having modified kmod
> ebuilds in my private overlay (poly-c) but of course that's no solution once
> we want to have braindamaged-udev in tree. One solution to this problem
> might be another package (e.g. liberated-kmod) that simply uses the kmod
> sources and installs all necessary files into / as well.
> To be honest it's about time to start working against this asinine / -->
> /usr merge that the systemd/udev/kmod fanatics started to impose on us. 
> That's my main motivation behind getting braindamaged-udev into the tree.

There is something else to consider as well. gen_usr_ldscript has had a bi-product of making it possible for us to have separate /usr without initramfs for longer than most distros. Toolchain has a bug open to make it a no-op on linux, and if they do that, libraries we are moving from /usr to / will be moved back to /usr, so anything that links against them will have to be put in /usr. gen_usr_ldscript wasn't put in place for udev, it was put in place because of bug #4411.

For the record also, I for one, have never said that I  am trying to force things onto gentoo, I just haven't found a way to *not* support the /usr merge.

If the kernel team gets behind an alternate udev, I have no problem with supporting/maintaining it. The only thing I am waiting for is to find out what they actually do.
Comment 24 William Hubbs gentoo-dev 2012-10-15 17:19:01 UTC
Unstable udev and kmod also will not be going to stable until I hear something about what the kernel guys are going to support.
Comment 25 grey dot 2012-10-15 17:32:36 UTC
(In reply to comment #23)
> There is something else to consider as well. gen_usr_ldscript has had a
> bi-product of making it possible for us to have separate /usr without
> initramfs for longer than most distros. Toolchain has a bug open to make it
> a no-op on linux, and if they do that, libraries we are moving from /usr to
> / will be moved back to /usr, so anything that links against them will have
> to be put in /usr. gen_usr_ldscript wasn't put in place for udev, it was put
> in place because of bug #4411.
> 
> For the record also, I for one, have never said that I  am trying to force
> things onto gentoo, I just haven't found a way to *not* support the /usr
> merge.
> 
> If the kernel team gets behind an alternate udev, I have no problem with
> supporting/maintaining it. The only thing I am waiting for is to find out
> what they actually do.

Despite that the actual shared library is installed in /lib, there is still a symlink at /usr/lib, so gen_usr_ldscript shouldn't be an issue. 

> ls -l /usr/lib/libkmod.so 
lrwxrwxrwx 1 root root 28 Sep 16 12:10 /usr/lib/libkmod.so -> ../../lib64/libkmod.so.2.2.0

Static libraries also go to /usr/lib, since they do not have anything to do with system boot.

(In reply to comment #24)
>Unstable udev and kmod also will not be going to stable until I hear something about what the kernel guys are going to support.

Stable is fine as it is right now with 171, I suppose. Though some changes might be backported as patches to support extra hardware (there is some s390 stuff for example).
Comment 26 William Hubbs gentoo-dev 2012-10-17 02:59:55 UTC
If you are using kmod from the tree, libkmod is stored in /usr/lib.
Comment 27 grey dot 2012-10-17 07:25:48 UTC
(In reply to comment #26)
> If you are using kmod from the tree, libkmod is stored in /usr/lib.

No, we provide a ebuild through udev overlay that installs libkmod shared library to /lib.

https://bitbucket.org/braindamaged/udev-overlay/src/baddd603cb2f66f1290465f802c08fce3878c3a4/sys-apps/kmod/kmod-10.ebuild?at=master

No issues whatsoever.
Comment 28 William Hubbs gentoo-dev 2012-10-26 15:59:54 UTC
(In reply to comment #27)
> (In reply to comment #26)
> > If you are using kmod from the tree, libkmod is stored in /usr/lib.
> 
> No, we provide a ebuild through udev overlay that installs libkmod shared
> library to /lib.
> 
> https://bitbucket.org/braindamaged/udev-overlay/src/
> baddd603cb2f66f1290465f802c08fce3878c3a4/sys-apps/kmod/kmod-10.
> ebuild?at=master
> 
> No issues whatsoever.

Something was pointed out to me that is in the actual tree that you might want to look into.

sys-apps/busybox-1.20.1, which is stable, supports a sep-usr use flag. If you set that use flag and emerge busybox, follow the enstructions you get in the elog. That gives an alternative way to mount /usr early without using an initramfs if you just want a simple setup. Note that this knows nothing about rade, lvm2, encryption, etc, so if you have /usr in one of those setups,  you would still need an initramfs.

Once that's done, it doesn't matter where things are installed, so there is no reason to move anything to /.
Comment 29 grey dot 2012-10-27 15:41:43 UTC
(In reply to comment #28)
> Something was pointed out to me that is in the actual tree that you might
> want to look into.
> 
> sys-apps/busybox-1.20.1, which is stable, supports a sep-usr use flag. If
> you set that use flag and emerge busybox, follow the enstructions you get in
> the elog. That gives an alternative way to mount /usr early without using an
> initramfs if you just want a simple setup. Note that this knows nothing
> about rade, lvm2, encryption, etc, so if you have /usr in one of those
> setups,  you would still need an initramfs.
> 
> Once that's done, it doesn't matter where things are installed, so there is
> no reason to move anything to /.

In another words, you're suggesting a huge kludge to fix this issue instead of fixing the actual cause in the first place, am I right?
Comment 30 William Hubbs gentoo-dev 2012-10-29 19:08:32 UTC
(In reply to comment #29)
> (In reply to comment #28)
> > Something was pointed out to me that is in the actual tree that you might
> > want to look into.
> > 
> > sys-apps/busybox-1.20.1, which is stable, supports a sep-usr use flag. If
> > you set that use flag and emerge busybox, follow the enstructions you get in
> > the elog. That gives an alternative way to mount /usr early without using an
> > initramfs if you just want a simple setup. Note that this knows nothing
> > about rade, lvm2, encryption, etc, so if you have /usr in one of those
> > setups,  you would still need an initramfs.
> > 
> > Once that's done, it doesn't matter where things are installed, so there is
> > no reason to move anything to /.
> 
> In another words, you're suggesting a huge kludge to fix this issue instead
> of fixing the actual cause in the first place, am I right?

Udev has nothing to do with separate /usr support. Yes, I will give you that udev is what has brought the issue to the forefront this time around, but this isn't the first time we have had issues with separate /usr.

If we had told people using separate /usr to start using initramfs a decade ago, we would not have had to come up with gen_usr_ldscript, and we wouldn't have had to move libraries from /usr/lib to /lib. Furthermore, if we had done that, we wouldn't be having this conversation about udev.

Other distros (specifically I am thinking of Archlinux and Debian) have been using initramfs to solve these issues for a long time.

In spite of being able to use genkernel to build an initramfs easily on gentoo, there are users who do not want to use one. This is why we have the
 busybox/sep-usr option.
Comment 31 grey dot 2012-10-30 05:23:41 UTC
(In reply to comment #30)
> Udev has nothing to do with separate /usr support. Yes, I will give you that
> udev is what has brought the issue to the forefront this time around, but
> this isn't the first time we have had issues with separate /usr.

Udev has everything to do with separate /usr support, since it is required to find and mount /usr (creating symlinks for lvm/raid devices, loading NIC firmware so we could mount /usr via NFS, etc).

> If we had told people using separate /usr to start using initramfs a decade
> ago, we would not have had to come up with gen_usr_ldscript, and we wouldn't
> have had to move libraries from /usr/lib to /lib. Furthermore, if we had
> done that, we wouldn't be having this conversation about udev.

If you had told? Mister Hubbs, stop _telling_ people what they should do and start doing _your_ job as a maintainer/developer. If everybody would have been doing everything right, we wouldn't have had this conversation in the first place. But since many people do not understand that boot-time software must not use resources which might not be available at a boot time, there is this (and many other) bug.

As for your gen_usr_ldscripts, maybe fixing the toolchain instead of creating another bunch of ugly kludges would do it, huh?

> Other distros (specifically I am thinking of Archlinux and Debian) have been
> using initramfs to solve these issues for a long time.
> 
> In spite of being able to use genkernel to build an initramfs easily on
> gentoo, there are users who do not want to use one. This is why we have the
>  busybox/sep-usr option.

Your busybox/sep-usr is a failure and is unusable in far too may cases to be actually useful somehow. Even statically built udev is more adequate solution here.

Anyway, it's been three weeks since this bug has been opened, and I'd like to know gentoo dev team position on this - we stay with a separate overlay, or import this into a main tree, or what?
Comment 32 William Hubbs gentoo-dev 2012-10-30 14:14:58 UTC
(In reply to comment #31)
> > If we had told people using separate /usr to start using initramfs a decade
> > ago, we would not have had to come up with gen_usr_ldscript, and we wouldn't
> > have had to move libraries from /usr/lib to /lib. Furthermore, if we had
> > done that, we wouldn't be having this conversation about udev.
> 
> If you had told? Mister Hubbs, stop _telling_ people what they should do and
> start doing _your_ job as a maintainer/developer. If everybody would have
> been doing everything right, we wouldn't have had this conversation in the
> first place. But since many people do not understand that boot-time software
> must not use resources which might not be available at a boot time, there is
> this (and many other) bug.

I wasn't a developer back then so I'm just pointing out another issue that makes separate /usr without an initramfs difficult at best.

> As for your gen_usr_ldscripts, maybe fixing the toolchain instead of
> creating another bunch of ugly kludges would do it, huh?
> 
> > Other distros (specifically I am thinking of Archlinux and Debian) have been
> > using initramfs to solve these issues for a long time.

I'm not a toolchain developer, so I can't really comment on the toolchain, but I can tell you that, like I said above, using initramfs is the most common way of dealing with issues like this.

> > In spite of being able to use genkernel to build an initramfs easily on
> > gentoo, there are users who do not want to use one. This is why we have the
> >  busybox/sep-usr option.
> 
> Your busybox/sep-usr is a failure and is unusable in far too may cases to be
> actually useful somehow. Even statically built udev is more adequate
> solution here.

Can you give me cases where statically built udev fixes the issue and either busybox[sep-usr] or initramfs do not?

> Anyway, it's been three weeks since this bug has been opened, and I'd like
> to know gentoo dev team position on this - we stay with a separate overlay,
> or import this into a main tree, or what?

There is still a lot of information I want to have before we attempt to import this into the tree.

I want to see what develops with  udev itself. If it is completely taken over and moved into kernel space like firmware loading has been in linux-3.7, we may not need a udev at all.

Your changes to kmod and xz-utils only change the install locations. We could do something like this with a use flag in the current in-tree ebuilds. If we do that, importing the ebuilds from your overlay is not necessary.

On the other hand, if the council agrees, as the qa lead does, that the sep-usr use flag on busybox and initramfs are the ways we should support separate /usr configurations, this isn't needed.

I will update this bug again as soon as I have more information.
Comment 33 grey dot 2012-10-30 15:20:39 UTC
(In reply to comment #32)
> I wasn't a developer back then so I'm just pointing out another issue that
> makes separate /usr without an initramfs difficult at best.

For some people learning something at pre-school is difficult. In fact, as you can see, everything is pretty easy.

> I'm not a toolchain developer, so I can't really comment on the toolchain,
> but I can tell you that, like I said above, using initramfs is the most
> common way of dealing with issues like this.

I'm not a udev developer, so I can't really... wait, OSHI~

> Can you give me cases where statically built udev fixes the issue and either
> busybox[sep-usr] or initramfs do not?

Mdev (busybox part) cannot load my nic firmware and mount /usr via NFS. Is this example enough? Again, we are not talking about total initramfs elimination here. Initramfs is useful in many cases, but in this particular one it is totally redundant.

> There is still a lot of information I want to have before we attempt to
> import this into the tree.
> 
> I want to see what develops with  udev itself. If it is completely taken
> over and moved into kernel space like firmware loading has been in
> linux-3.7, we may not need a udev at all.

Right now it is not taken over and is not moved into kernel space, and I do not see any related commits in kernel source tree.

> Your changes to kmod and xz-utils only change the install locations. We
> could do something like this with a use flag in the current in-tree ebuilds.
> If we do that, importing the ebuilds from your overlay is not necessary.

And I could save the world. Talk is cheap, Mister Hubbs, show me some action.
Comment 34 Khayyam 2012-11-01 16:24:58 UTC
While seperate /usr, and the discussion of the relative merits of the methods to continue to support it, is important, this is somewhat of a seperate issue to the inclusion of "udev standalone" in the tree. In fact, I don't see this as being the key issue involved. As I quoted above, udev-systemd upstream has stated that "udev on non-systemd systems is in our eyes a dead end" and further stated that they are "looking forward to the day when [they] can drop that support entirely" (Lennart Poettering, systemd-devel mailing list August 1st 2012). I am far more concerned with this move, and systemd's coup d'main of init, than with any issues relating to seperate /usr.

What might happen in the linux kernel is currently an unknown, but as devtempfs has systemd developers behind it I'm inclined to think that they will do their best to keep as tight a grip on dev management as they can, and further systemd as the defacto init for linux. As things are shaping up there are now higher level OS components which require systemd (as a "blessed dependency") and I don't imagine it'll stop unless there is some serious oposition, and the options to avoid it are available, 'standalone udev' is one (perhaps small) step in that direction.

I worry that what is essencially happening is a process by which systemd will arrive at its goal by stealth, all avenues and methods of avoiding it having been blocked. Systemd is finding its way into gentoo by virtue of the principle of "user choice", but as it brings with it a narrowing of choice, and nothing less than the corralling of choice into the path laid out by systemd developers, I can't help but see this a negative and stealthy incursion.

While it is no doubt a difficult problem, and there is much that is out of gentoo's control, I think that its not yet a done deal, and so I'm not yet ready to abandon hope, but things need to happen, and a non-systemd udev is one small step in the (imo) right direction. So, lets not let this get bogged down in distractions, the key issue is how/if gentoo will be capable of maintaing "user choice" when upstream are corralling that choice into the narrowly circumscribed decisions made by systemd developers (and by extention, Redhat).

best ... khay
Comment 35 Weedy 2012-11-11 23:26:32 UTC
> sys-fs/udev-standalone
> block on sys-fs/udev
> provide sys-fs/udev
> post-install note about INSTALL_MASK="/lib/systemd /usr/lib/systemd/system" or something

I'm not seeing how this is a huge deal or why we are bike sheading so much.

I would love to see gentoo take a stand agaist this shit systemd devs are doing, especially since we lost Arch to that camp.
Comment 36 Kevin Toppins 2012-11-13 06:42:23 UTC
I am just a Gentoo user, not a developer.

I have nothing of technical merit to offer here, and I am (virtually) completely uninformed compared to others here.

However, I started using Gentoo in 2006, and have struggled through the years to ascend the endless learning curves.

I, to this day, have never left Gentoo.

I feel that - in principle - Gentoo has it 100 % RIGHT.

You have a choice, even when a lot of the time it's choosing to learn about what not to do, but you _have_ freedom.

Apple is/has been locking down iPods from open source programs.

MicroSoft is shutting the gates with the Windows 8 Store.

This stupid crap of "Everybody should have the freedom to choose, as long as that choice is Systemd (TM)" is another example of people in some position of power trying to consolidate that power.

It is Human nature - but that is not a justification for it to stay that way.

Linux dissolved UNIX's power hold.

GPL dissolved copyright's lines in drawn in the sand.

The Systemd group will be denied, continuously, a position of converging power if - in the hierarchy of things - a path around them is formed.

That's all it takes - at least, to start.

Even if everything else is severely not in perfect working order, this is - in essence - an evolutionary process.

An independent, alternative, and modern device manager - in whatever form or name it takes - is needed to stop certain people from strangling the rest of us.

So, I guess this is a rant, but it is a rant of gratitude for things you have done, and the things you continue to do - and a bit of encouragement to keep going.


I am almost certain that a very large group of developers - currently disparate and unassociated - would much rather chip in to help an alternative device manager get situated in Linux -> than devolve their respective code to be Borg assimilated into Systemd.

But until it gets started - at the very least, an unstable package in the main Gentoo tree - a vast army of support will never come its way.

 : D

And yeah, that ...   yeah.

ADD meds != helping.
Comment 37 Dale 2012-11-14 22:00:48 UTC
Some are talking about a name for this fork.  I propose smartdev.  I think that name says that it is different, device related and not confusing such as sudev and others that have been mentioned.  

Also, thanks much to the people that are working on this.  Seriously, THANKS !!!
Comment 38 Anthony Basile gentoo-dev 2012-11-16 04:21:54 UTC
(In reply to comment #1)
> Can you link to it at least?

We are working on an alternative fork which tries to be closer to the latest upstream udev while still stripping out systemd.  We will also try to remain neutral with respect to 1) system initialization, so openrc should be fine and 2) choice of modutil.   I'm also trying to rid the udev code of any glibc-ism.

  https://github.com/gentoo/udev-ng


We here = me, axs, chainsaw, ryao
Comment 39 Sander Sweers 2012-11-16 16:06:50 UTC
(In reply to comment #38)
> We are working on an alternative fork which tries to be closer to the latest
> upstream udev while still stripping out systemd.  We will also try to remain
> neutral with respect to 1) system initialization, so openrc should be fine
> and 2) choice of modutil.   I'm also trying to rid the udev code of any
> glibc-ism.
> 
> We here = me, axs, chainsaw, ryao

Now we have two forks basically doing the same thing. May I suggest the two groups get together and see if they can merge the 2 projects and pool the resources?
Comment 40 consus 2012-11-16 17:02:26 UTC
> We are working on an alternative fork which tries to be closer to the latest
> upstream udev 

Just curious: what features are missing in our fork?
Comment 41 grey dot 2012-11-16 18:21:33 UTC
(In reply to comment #38)
> (In reply to comment #1)
> > Can you link to it at least?
> 
> We are working on an alternative fork which tries to be closer to the latest
> upstream udev while still stripping out systemd.  We will also try to remain
> neutral with respect to 1) system initialization, so openrc should be fine
> and 2) choice of modutil.   I'm also trying to rid the udev code of any
> glibc-ism.
> 
>   https://github.com/gentoo/udev-ng
> 
> 
> We here = me, axs, chainsaw, ryao

It's just what we have already done - removed all systemd/libsystemd references and ported significant changes. Could you please contact me so we could talk this out? The idea of having two forks freaks me out.
Comment 42 Markos Chandras (RETIRED) gentoo-dev 2012-11-17 11:34:51 UTC
yeah that would make sense. Since all of you are working towards the same goal, at list try to make it right and not start the fork of the fork of the fork from the very beginning
Comment 43 Richard Yao gentoo-dev 2012-11-20 12:57:12 UTC
(In reply to comment #41)
> It's just what we have already done - removed all systemd/libsystemd
> references and ported significant changes. Could you please contact me so we
> could talk this out? The idea of having two forks freaks me out.

Our goals go far beyond that. We want better compatibility with existing software. i.e. older kernels, various libcs, various init/RC systems, and different userlands.

Kay Sievers et al have done a good enough job of burning bridges that systemd will not be upstream for us any more than NetBSD is upstream for OpenBSD. However, we intend to watch systemd's GIT for changes that could harm compatibility. Our principle goal is for eudev to be a drop-in replacement for systemd-udev that is as compatible as possible. 

With that said, we would be happy to work toward the merger of our projects, but please understand that our goals are not strictly the same.
Comment 44 Weedy 2012-11-21 07:03:29 UTC
(In reply to comment #43)
> (In reply to comment #41)
> > It's just what we have already done - removed all systemd/libsystemd
> > references and ported significant changes. Could you please contact me so we
> > could talk this out? The idea of having two forks freaks me out.
> 
> Our goals go far beyond that. We want better compatibility with existing
> software. i.e. older kernels, various libcs, various init/RC systems, and
> different userlands.
> ...
> With that said, we would be happy to work toward the merger of our projects,
> but please understand that our goals are not strictly the same.

What ever happens can we set a date? like Dec 1st. To have something either commited to the tree (preferibly) or a clear direction?

This is way past annoying.
Comment 45 Kevin Toppins 2012-11-21 07:45:09 UTC
Just posted this.

It probably doesn't go here or there, but I think I make some points that are helpful to consider.

I would be pleased if you read them because I think they help sharpen our focus of what is going on here.

3. http://lists.debian.org/debian-devel/2012/11/msg00563.html

That being the last in a series of 3. Posts 1 and 2 are to help explain where did 3 come from. 

1. http://lists.debian.org/debian-devel/2012/11/msg00469.html

2. http://lists.debian.org/debian-devel/2012/11/msg00485.html


Please don't dismiss them because they link to debian and this is gentoo.

Please don't dismiss them because they don't seem to revolve around udev very concretely, this is much bigger and much more important.

Please just read them, consider what they say, and hopefully it will then make sense how it also would help to say them here.


-Kev
Comment 46 Kevin Toppins 2012-11-21 08:05:21 UTC
I forgot to mention.....

To me, this looks like systemd (and familyd) is trying to assert *itself* as the *linux kernel*.

This looks very much like a coup of the authority structure of the GNU/Linux architecture.

We need to stop this now if it is, and that udev fork is the key to doing it.

So please don't stop.

I know I don't submit code, and I respect that you do not have infinite free time.

So....

 -> First off, thanks.

 -> I think this udev fork has now become *really essential*.

 -> I think this udev fork is the key to stopping this madness.

-Kev
Comment 47 Richard Yao gentoo-dev 2012-11-22 10:02:34 UTC
(In reply to comment #44)
> (In reply to comment #43)
> > (In reply to comment #41)
> > > It's just what we have already done - removed all systemd/libsystemd
> > > references and ported significant changes. Could you please contact me so we
> > > could talk this out? The idea of having two forks freaks me out.
> > 
> > Our goals go far beyond that. We want better compatibility with existing
> > software. i.e. older kernels, various libcs, various init/RC systems, and
> > different userlands.
> > ...
> > With that said, we would be happy to work toward the merger of our projects,
> > but please understand that our goals are not strictly the same.
> 
> What ever happens can we set a date? like Dec 1st. To have something either
> commited to the tree (preferibly) or a clear direction?
> 
> This is way past annoying.

A date has already been set. December 1st is our target for the first tag. We don't have much more time thanks to the deadline that chainsaw set for us in the Gentoo Council meeting.

As for having it in the tree, I want to have the 9999 ebuild in the tree before our first tag. AxS and I are working on the ebuild in parallel to blueness who has been working on our build system. The plan is for the ebuild to go into the tree when it can be used as part of a working system without any work beyond the standard emerge and etc-update commands.
Comment 48 Richard Yao gentoo-dev 2012-11-23 07:43:09 UTC
I have opened a bug to discuss how virtual/udev will enter the tree. Once it is in the tree, it should be possible to bring this fork into the tree as well, provided that a developer volunteers to maintain it.
Comment 49 Khayyam 2012-11-23 12:19:43 UTC
(In reply to comment #48)
> I have opened a bug to discuss how virtual/udev will enter the tree. Once it
> is in the tree, it should be possible to bring this fork into the tree as
> well, provided that a developer volunteers to maintain it.

Richard ... doesn't virtual/dev-manager serve this purpose currently? The virtual includes:

sys-fs/udev
sys-apps/busybox[mdev]
sys-fs/devfsd
sys-fs/static-dev
sys-freebsd/freebsd-sbin

to which could be added:

sys-fs/eudev
sys-fs/<whatever_this_fork_is_eventually_named>

best ... khay
Comment 50 Richard Yao gentoo-dev 2012-11-23 23:33:12 UTC
(In reply to comment #49)
> (In reply to comment #48)
> > I have opened a bug to discuss how virtual/udev will enter the tree. Once it
> > is in the tree, it should be possible to bring this fork into the tree as
> > well, provided that a developer volunteers to maintain it.
> 
> Richard ... doesn't virtual/dev-manager serve this purpose currently? The
> virtual includes:
> 
> sys-fs/udev
> sys-apps/busybox[mdev]
> sys-fs/devfsd
> sys-fs/static-dev
> sys-freebsd/freebsd-sbin
> 
> to which could be added:
> 
> sys-fs/eudev
> sys-fs/<whatever_this_fork_is_eventually_named>
> 
> best ... khay

Certain packages cannot work with alternative virtual/dev-manager providers. At present, that is handled through an explicit dependency on sys-fs/udev, but if we are going to accommodate multiple forks, we would need to update the dependencies of every package in the tree to handle them. A virtual would permit us to do this only once while the alternative is to do it every single time.

In addition, any overlays providing udev forks presently must replace the existing sys-fs/udev package. Abusing overlays in this way is fragile because it depends on staying ahead of the sys-fs/udev package in the main tree to avoid switching to the main tree's fork. It also does not scale to permit multiple udev forks to co-exist in the main tree. virtual/udev should solve those problems.
Comment 51 Kevin Toppins 2012-11-24 03:59:44 UTC
I have an idea.

It might be a terrible idea. I could use your greater knowledge of udev to analyze its merits.

Can you consider it, and then see if you think it worth trying?

I don't know much about udev, so I might not be helping here. Sorry, if that is the case.

 All the udev forks share one thing in common (I think, not completely sure).

 -> They all *diverged* from systemd at some point.

Can the point where each fork broke away from systemd be mapped? (not path, but just point)

They all have some difference between systemd at the time of the fork, and the fork itself right when it broke.

I'm not talking about how are they different now, that would be way harder to identify, and is why we still have fragmented pieces.

If we can examine these *fundamental* differences between fork <-> systemd at the *moment* they diverged, it might be possible to start mapping out an abstraction of what they are all doing. They are all implementing some model of udev. They should not be fundamentally incongruent from one another. I think it's surface details that make up the fragmentation.

You could consider, systemd removed some piece right before each fork, we might be able to see what it was and identify the mechanism that stopped working in systemd and showed up in the fork.

Identifying what mechanism(s) each fork reimplemented might allow us to abstract a *logical map* of the mechanisms.

All of the mechanisms, when you abstract and reduce them, might reduce down to the same logical mechanism. Being able to identify what that is would be enourmously useful to re-integrating the forks.


If we keep the forks in place, would that not stop improvements to udev? -> Because improving udev is really improving a handful of forks of udev, all of them in a different way.


So... is even remotely close to a workable idea?

If it's clear -> I don't know what I am talking about when it comes to udev, you can just say so, it will not be taken with offence.

 : D



-Kev
Comment 52 consus 2012-11-25 15:30:27 UTC
(In reply to comment #48)
> I have opened a bug to discuss how virtual/udev will enter the tree. Once it
> is in the tree, it should be possible to bring this fork into the tree as
> well, provided that a developer volunteers to maintain it.

So yeah, I volunteer. Where to sign?
Comment 53 Kevin Toppins 2012-11-27 13:32:17 UTC
Guys, sorry to bother you, but I could _really_ use your help to spread this message.

Not only is systemd a _bad_ idea, it is also _dangerous_ to do, now that computers are integrated with technology. I show how stupid programming killed people when Air France 447 crashed in 2009.

I try to reason this out with people over at debian-devel.

I think I make a _damn_ solid case.

Yes, it is a bit of a read.


 -> Yes, it is that important.


Please help _spread_ the word for people to read this.

http://lists.debian.org/debian-devel/2012/11/msg00670.html

-Kev
Comment 54 Markos Chandras (RETIRED) gentoo-dev 2012-11-27 13:40:05 UTC
(In reply to comment #53)
> Guys, sorry to bother you, but I could _really_ use your help to spread this
> message.
> 
> Not only is systemd a _bad_ idea, it is also _dangerous_ to do, now that
> computers are integrated with technology. I show how stupid programming
> killed people when Air France 447 crashed in 2009.
> 
> I try to reason this out with people over at debian-devel.
> 
> I think I make a _damn_ solid case.
> 
> Yes, it is a bit of a read.
> 
> 
>  -> Yes, it is that important.
> 
> 
> Please help _spread_ the word for people to read this.
> 
> http://lists.debian.org/debian-devel/2012/11/msg00670.html
> 
> -Kev

Hi,

Bugzilla is not the place to seek help for such things. Please don't hijack this bug with (semi-)irrelevant stuff.
Comment 55 Kevin Toppins 2012-11-27 13:43:58 UTC
I understand that.

I would not _normally_ do this.

However, I think this is important enough to justify asking for help, and the people here have some understanding of just how screwed up systemd is.

So, I am sorry this seems out of place, but _please_ just read it before judging that is not important.

-Kev
Comment 56 Kevin Toppins 2012-11-27 14:15:41 UTC
Just so it helps to convey.

The seriousness of the points I am making increases with each post.

The first seeming to have little importance.

The 4th, 5th, 6th, and 7th honing in on a major problem if people adopt systemd.

Hopefully that will help clarify why I am asking.

-Kev
Comment 57 grey dot 2012-11-27 14:26:08 UTC
(In reply to comment #56)
> Just so it helps to convey.
> 
> The seriousness of the points I am making increases with each post.
> 
> The first seeming to have little importance.
> 
> The 4th, 5th, 6th, and 7th honing in on a major problem if people adopt
> systemd.
> 
> Hopefully that will help clarify why I am asking.
> 
> -Kev

Please start a forum thread to discuss this issue. Bugzilla serves other purpose. Btw there are too many stupid things in linux to deal with.
Comment 58 Richard Yao gentoo-dev 2012-11-27 22:52:25 UTC
(In reply to comment #52)
> (In reply to comment #48)
> > I have opened a bug to discuss how virtual/udev will enter the tree. Once it
> > is in the tree, it should be possible to bring this fork into the tree as
> > well, provided that a developer volunteers to maintain it.
> 
> So yeah, I volunteer. Where to sign?

Once the mass conversion of all ebuilds to rely on virtual/udev is done, it should be possible to bring your fork into the tree, possibly as sys-fs/budev. If you produce an ebuild that is ready for inclusion, the eudev team will proxy commit for you after the dependency situation has been resolved.

I will reassign this bug to the eudev team once our email alias is created.
Comment 59 consus 2012-11-28 07:00:10 UTC
Great. Thanks.
Comment 60 grey dot 2012-12-28 05:23:59 UTC
I withdraw my request since we decided to abandon the project.