Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 368597 - stage3-amd64-20110520 - no devices present
Summary: stage3-amd64-20110520 - no devices present
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Release Media
Classification: Unclassified
Component: Stages (show other bugs)
Hardware: AMD64 Linux
: Normal normal with 4 votes (vote)
Assignee: Gentoo Release Team
URL:
Whiteboard:
Keywords:
: 369873 370733 371263 371787 372535 374179 (view as bug list)
Depends on: 369085
Blocks:
  Show dependency tree
 
Reported: 2011-05-24 14:40 UTC by Claude Juif
Modified: 2011-07-09 16:57 UTC (History)
21 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
devices.patch (devices.patch,669 bytes, text/plain)
2011-05-28 20:21 UTC, William Hubbs
Details
devices.patch (devices.patch,1.04 KB, text/plain)
2011-05-29 23:03 UTC, William Hubbs
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Claude Juif 2011-05-24 14:40:57 UTC
Hi,

It seems that current stage3 tarball is missing files in /dev. The only device file is null. So my kernel fail to boot as it has no /dev/console device.

I'm not sure if this is really a bug because many things have changed in this stage compare to gentoo handbook.

So, it may be a bug in this stage3 tarball or gentoo handbook not up to date.

Reproducible: Always

Steps to Reproduce:
1. Follow Gentoo Handbook for amd64 until you uncompress stage3 tarball
2. ls /mnt/gentoo/dev only return null
3.
Actual Results:  
At the end of the handbook when i have to reboot my kernel stop and do nothing because my root fs was missing /dev/console device

Expected Results:  
/dev/console should be present in the stage3 tarball

in the previous stage3 tarball, we can find any device (stage3-amd64-20110428).

If this is a feature, can you provide me a link to the right handbook ? 

Today, i think this is critical because you can't install gentoo Linux with this stage follwoing the handbook.
Comment 1 Claude Juif 2011-05-24 14:50:45 UTC
For people who have this problem, a possible workaround is :
 - cp -dpRL /dev/{console,kmem,mem,null,urandom,random,zero,ptmx,ram[0-6],tty[0
-6]} /mnt/gentoo/dev 
before you mount /dev filesystem (between tar xvjf stage3-*.tar.bz2 and mount --rbind /dev /mntgentoo/dev)
Comment 2 Claude Juif 2011-05-24 14:53:44 UTC
Corrected one (sorry for typo)

For people who have this problem, a possible workaround is :
 - cp -dpRL /dev/{console,kmem,mem,null,urandom,random,zero,ptmx,ram[0-6],tty[0-6]} /mnt/gentoo/dev 
before you mount /dev filesystem (between tar xvjpf stage3-*.tar.bz2 and mount --rbind /dev /mnt/gentoo/dev)
Comment 3 Patryk Rzadzinski 2011-05-24 16:18:49 UTC
Confirmed here too, took me a whole day to realize this is missing ;(
Comment 4 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-05-24 22:02:35 UTC
Can one of you please run the following test?

Use the livecd to get to your new install and make sure you only have /dev/null in the new system. Configure your new kernel and be sure to set to Y the  CONFIG_DEVTMPFS and CONFIG_DEVTMPFS_MOUNT options in the kernel. Build the kernel and boot it.
Is your system able to boot or does it still complain about missing devices?
Comment 5 Claude Juif 2011-05-25 00:08:01 UTC
Hi,

Yes CONFIG_DEVTMPFS and CONFIG_DEVTMPFS_MOUNT fix the problem.

Bug is not related to stage3 tarball. However i think that there is something too "new" here because livecd kernel does not come with devtmpfs builtin (we use it for genkernel) and handbook don't talk about this option. (in previous stage3 tarball /dev use to be filled with a lot of devices)

Maybe this bug report should be move to Documentation ??

Many thanks for your help,
Comment 6 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-05-25 02:22:41 UTC
@releng, @docs, @base-system,

how shall we address this?
If we don't want to provide the device nodes through baselayout, bl1 called makedev, then we'll have to ask the docs-team to add a strong warning in the handbook for the required options for the kernel. If we don't want to risk users failing to boot a system for missing the device nodes, then we'll have to get them in the stages.
What do you say?
Comment 7 nm (RETIRED) gentoo-dev 2011-05-25 06:34:25 UTC
We definitely don't want to document workarounds in the handbook. The stages themselves should just work. If stuff is broken, it should be fixed on the installed files side, not the docs side.
Comment 8 William Hubbs gentoo-dev 2011-05-25 07:51:11 UTC
I need to know something about how the udev startup scripts work.

If you turn on CONFIG_DEVTMPFS do you need CONFIG_DEVTMPFS_MOUNT as
well

If you do not turn on either of these, how is /dev populated?
Comment 9 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-05-25 12:11:14 UTC
As Diego just pointed out to me, the stage includes a /dev/null regular file, not a device file. I've checked on my local stages and this is what I have:

-rw-r--r-- root/root              0 2011-05-08 04:19 ./dev/null
Comment 10 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-05-25 12:14:07 UTC
(In reply to comment #7)
> We definitely don't want to document workarounds in the handbook. The stages
> themselves should just work. If stuff is broken, it should be fixed on the
> installed files side, not the docs side.

Josh,

the stages aren't "broken" by themselves. It might be argued by base-system that we shouldn't have a static /dev list provided by baselayout.
More importantly, if we go down this route, what we need is to tell users about required kernel options. This will be not a workaround, but a requirement for installations.
Comment 11 William Hubbs gentoo-dev 2011-05-25 17:26:29 UTC
udev maintainers,

can udev be set up to populate /dev even if  devtmpfs is not mounted?
Comment 12 SpanKY gentoo-dev 2011-05-25 19:34:15 UTC
i dont see what udev has to do with things.  the stage3 tarball must have /dev/null and /dev/console in it.  devtmpfs is unrelated.

the warnings describe happen before userspace is even executed.  so anything involving userspace isnt relevant.
Comment 13 William Hubbs gentoo-dev 2011-05-25 20:43:11 UTC
(In reply to comment #12)
> i dont see what udev has to do with things.  the stage3 tarball must have
> /dev/null and /dev/console in it.  devtmpfs is unrelated.

Are these two the only devices a stage 3 tarball needs?
Comment 14 Fernando (likewhoa) 2011-05-25 20:53:32 UTC
as #12 mentioned, the only required device nodes are /dev/{null,console} for proper boot as udev will create the rest of the nodes on init start, furthermore Genkernel automatically creates these nodes within the initramfs but for a stage3 only those two are require.
Comment 15 Claude Juif 2011-05-26 08:09:19 UTC
Not sure if this is relevant, but before trying CONFIG_DEVTMPFS, i've tried genkernel who complain too about missing /dev/console.

So I'm not sure that genkernel automatically create those devices in initramfs.

By the way creating thos 2 devices (console and null) everything is working as expected.

Many thanks for all the reply, i'm learning many things about gentoo ;)
Comment 16 Claude Juif 2011-05-26 14:09:41 UTC
After reinstalling my gentoo box with same stage tarball, i've tried to only create the two device node (null and console), but my system fail to start wih this error : 

Could not create /dev/pts!
Could not create /dev/shm!

And many errors like No such file or directory while trying to open /dev/sda5 (my root partition). It seems to me that udev is not properly launch or may be not launch at all. Any clue on how i can check which part of init is failing ?

So by now we must activate CONFIG_DEVTMPFS and CONFIG_DEVTMPFS_MOUNT to be able to boot Gentoo correctly, or copy all devices files.
Comment 17 Ben Kohler gentoo-dev 2011-05-26 14:21:17 UTC
I've tried this stage3 on a new VM, and I found that simply copying /dev/{null,console} onto your rootfs /dev is not enough.  This stage3 also doesn't have udev in sysinit runlevel so it never starts.  If I copy null & console, AND "rc-update add udev sysinit", it works.

Enabling CONFIG_DEVTMPFS and CONFIG_DEVTMPFS_MOUNT also "works", but udev still never actually takes over unless you explicitly add it to a runlevel.
Comment 18 William Hubbs gentoo-dev 2011-05-27 20:40:23 UTC
All,

I found the same issues in stage3-i686-20110524.

For the record, here is what I found:

/dev/null exists, but it is a regular file. Do we know how this file is
being created? If so, can we make the file be the device with the
correct major and minor numbers and create /dev/console the same way?

The issue with udev not being in the sysinit runlevel also exists. We
need to figure out how to get udev added to sysinit. Jorge suggested one
possibility for that would be to have the udev ebuild add  the symbolic
link that does this if the build use flag is active.
Comment 19 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-05-28 14:35:07 UTC
As this is the 2nd week we put out stages without devices and that is causing issues to users trying to install Gentoo, let's please try to fix this asap.
From the comments here and the discussions on IRC, I propose the following 2 changes (if you can see a problem with this, please let us know):

1. update bl2 and on pkg_postinst create the /dev/null, /dev/console for
USE="build".
2. update udev and for USE="build" create the /etc/runlevels/sysinit/ symlink.
Does this need to be done in pkg_postinst as well?

If no one can see any reason why the above won't work or any substantial reason not to do it, I'd like to ask both packages to be fixed asap!
Comment 20 SpanKY gentoo-dev 2011-05-28 15:31:11 UTC
/dev/{null,console} are the bare min required in the stage3 tarball

baselayout-1 would recover if /dev/console did not exist.  i dont know if openrc works around this though.  something to put into a new bug.

udev init.d script not launching is a different bug.  file a new one rather than dumping here.

baselayout-1 was responsible for seeding device nodes during USE=build, but that was migrated to sys-apps/makedev (as well as the MAKEDEV script).  baselayout-2 was depending on makedev, but it looks like someone incorrectly dropped it (Cardoe or Roy?) when creating the initial openrc ebuilds.

so we'll need arch peeps to stabilize sys-apps/makedev, and update sys-apps/openrc to add back sys-apps/makedev to the depend string for kernel_linux users.
Comment 21 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-05-28 16:44:44 UTC
(In reply to comment #20)
> /dev/{null,console} are the bare min required in the stage3 tarball
> 
> baselayout-1 would recover if /dev/console did not exist.  i dont know if
> openrc works around this though.  something to put into a new bug.
> 
> udev init.d script not launching is a different bug.  file a new one rather
> than dumping here.

bug 369037 created to track this issue

> baselayout-1 was responsible for seeding device nodes during USE=build, but
> that was migrated to sys-apps/makedev (as well as the MAKEDEV script). 
> baselayout-2 was depending on makedev, but it looks like someone incorrectly
> dropped it (Cardoe or Roy?) when creating the initial openrc ebuilds.
> 
> so we'll need arch peeps to stabilize sys-apps/makedev, and update
> sys-apps/openrc to add back sys-apps/makedev to the depend string for
> kernel_linux users.

@openrc:

Can we please get this done asap?
Comment 22 William Hubbs gentoo-dev 2011-05-28 19:04:03 UTC
(In reply to comment #21)
> (In reply to comment #20)
> > baselayout-1 was responsible for seeding device nodes during USE=build, but
> > that was migrated to sys-apps/makedev (as well as the MAKEDEV script). 
> > baselayout-2 was depending on makedev, but it looks like someone incorrectly
> > dropped it (Cardoe or Roy?) when creating the initial openrc ebuilds.
> > 
> > so we'll need arch peeps to stabilize sys-apps/makedev, and update
> > sys-apps/openrc to add back sys-apps/makedev to the depend string for
> > kernel_linux users.
> 
> @openrc:
> 
> Can we please get this done asap?

I checked makedev upstream, and they haven't done a release since  2006. Also, according to http://people.redhat.com/nalin/MAKEDEV/README.html, this utility is largely obsolete on modern systems because of udev.

So, I have another question. If we manually create /dev/null and /dev/console and add udev to the sysinit runlevel when USE=build, would that solve our issues?
Comment 23 SpanKY gentoo-dev 2011-05-28 20:04:27 UTC
last release date is irrelevant.  we (like many distros) were shipping a MAKEDEV that was maintained by hand in baselayout-1.  for obvious reasons, that's undesirable, so we switched to external makedev.

while one use of it has been obsoleted (historically people ran `MAKEDEV <category>` to seed their /dev with nodes), the other functions still work exactly the same even if udev or anything else is running.  the nodes it knows about have static major/minor numbers and naming, and so MAKEDEV still works fine with those.

it is also useful to have a certain "base" set of nodes on the rootfs for recovery purposes, which is also what sys-apps/makedev provides.

moving lists of device nodes to create into the openrc ebuild is not the way to go.
Comment 24 William Hubbs gentoo-dev 2011-05-28 20:21:51 UTC
Created attachment 274939 [details]
devices.patch

All,

If the answer to my question in comment #22 is yes, I think baselayout-2
should create these devices.  This patch will make baselayout-2 do this
if USE=build. I will commit this if there are no objections.

I am also in the process of becoming a udev maintainer, so
I will make that ebuild add the symbolic link to the sysinit runlevel if
USE=build.
Comment 25 SpanKY gentoo-dev 2011-05-28 21:01:19 UTC
as i said, this doesnt belong in baselayout.  it's already in makedev.  so simply fixing the DEPEND once makedev is stable will fix the stages.

also, your patch lacks proper quoting
Comment 26 William Hubbs gentoo-dev 2011-05-29 00:14:47 UTC
(In reply to comment #23)
> while one use of it has been obsoleted (historically people ran `MAKEDEV
> <category>` to seed their /dev with nodes), the other functions still work
> exactly the same even if udev or anything else is running.  the nodes it knows
> about have static major/minor numbers and naming, and so MAKEDEV still works
> fine with those.

The makedev ebuild will add 4280 device nodes to the stages in its current setup. I'm pretty sure we would not want all of those devices in the stages.

You can use MAKEDEV to make specific devices, e.g.

MAKEDEV -d "${ROOT}"/dev null
MAKEDEV -d "${ROOT}"/dev -x console

I think this is what we will want in the makedev ebuild so that we do not get all of those device nodes into the stages if we go this route.

> it is also useful to have a certain "base" set of nodes on the rootfs for
> recovery purposes, which is also what sys-apps/makedev provides.

I don't believe there is a need to use makedev to populate the /dev directory for the stages as long as we make sure to add udev to the sysinit runlevel. If we do that, /dev will be automatically populated by udev.

Am I wrong about that?
Comment 27 Christian Ruppert (idl0r) archtester Gentoo Infrastructure gentoo-dev Security 2011-05-29 00:19:00 UTC
(In reply to comment #26)
> (In reply to comment #23)
> > while one use of it has been obsoleted (historically people ran `MAKEDEV
> > <category>` to seed their /dev with nodes), the other functions still work
> > exactly the same even if udev or anything else is running.  the nodes it knows
> > about have static major/minor numbers and naming, and so MAKEDEV still works
> > fine with those.
> 
> The makedev ebuild will add 4280 device nodes to the stages in its current
> setup. I'm pretty sure we would not want all of those devices in the stages.
> 
> You can use MAKEDEV to make specific devices, e.g.
> 
> MAKEDEV -d "${ROOT}"/dev null
> MAKEDEV -d "${ROOT}"/dev -x console
> 
> I think this is what we will want in the makedev ebuild so that we do not get
> all of those device nodes into the stages if we go this route.
> 
> > it is also useful to have a certain "base" set of nodes on the rootfs for
> > recovery purposes, which is also what sys-apps/makedev provides.
> 
> I don't believe there is a need to use makedev to populate the /dev directory
> for the stages as long as we make sure to add udev to the sysinit runlevel. If
> we do that, /dev will be automatically populated by udev.
> 
> Am I wrong about that?

I would use mknod instead and not pull in another (in this case) useless dependency because mknod for null/console should be enough and as soon as they're existent udev will be able to populate the rest.

So IMHO:
Add udev to the sysinit runlevel and do mknod for null and console if USE=build, as Jorge already said.
Comment 28 SpanKY gentoo-dev 2011-05-29 16:57:13 UTC
the behavior of MAKEDEV is what we've always been doing.  so those many many nodes is exactly how we've been shipping baselayout-1.

saying "you must always run udev" to get a sane system isnt acceptable.  not only has that shown many times here to result in a flaky system, but it's shown in the past as well.  so simply creating the bare min doesnt make sense.  we want a sane base set sitting on the rootfs.

while 4000 nodes sounds like a lot, it takes up minuscule amount of diskspace.  although the fast majority of those are simply floppy disk and old style pty nodes ... killing them brings the set down to like ~250.

i can probably create a slightly tweaked set and have makedev use that.
Comment 29 SpanKY gentoo-dev 2011-05-29 18:16:07 UTC
makedev produces a tighter set now.  once it gets stabilized, tying it to baselayout makes more sense than openrc (since that's where it has always been part of, and we dont want to tie it to the init).  so i'll make that commit.

http://sources.gentoo.org/sys-apps/makedev/makedev-3.23.1.ebuild?r1=1.6&r2=1.7

udev issue is Bug 369037, so default runlevel handling can be taken up there.
Comment 30 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-05-29 21:11:57 UTC
It seems now that we have 2 ways to address the issue reported in this bug:

1. Create only /dev/{null,console} on baselayout-2
2. Have baselayout-2 use / depend on makedev to get the ~ 250 nodes.

In both cases we'll want to solve the udev issue on bug 369037, but any discussion about that should be done on that bug.

I like the idea of providing the minimal required devs and rely on udev, but if those 250 nodes will help users that make a small mistake in their systems configuration or will add support for users not relying on udev, that seems reasonable.

It feels like we've been going back and forward on this bug, so it would help if everyone clearly expressed their views. We also need to fix this asap so let's reach a decision and stick with it.
Comment 31 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-05-29 21:58:19 UTC
Following Mike's patch to makedev, with the following I got 252 nodes here (including /dev/console).

RCS file: /var/cvsroot/gentoo-x86/sys-apps/makedev/makedev-3.23.1.ebuild,v
retrieving revision 1.8
diff -u -b -B -r1.8 makedev-3.23.1.ebuild
--- makedev-3.23.1.ebuild       29 May 2011 20:03:12 -0000      1.8
+++ makedev-3.23.1.ebuild       29 May 2011 21:55:50 -0000
@@ -43,10 +43,10 @@
 pkg_postinst() {
        if use build ; then
                # set up a base set of nodes to make recovery easier #368597
-               MAKEDEV -d "${ROOT}"/dev hda input std sd tty
+               MAKEDEV -d "${ROOT}"/dev console hda input std sd tty
                # trim useless nodes
                rm -f "${ROOT}"/dev/fd[0-9]* # floppy
-               rm -f "${ROOT}"/dev/sd[a-d][a-z]* "${ROOT}"/dev/sd[e-z]* # excess sata
-               rm -f "${ROOT}"/dev/tty[a-z]* # excess tty
+               rm -f "${ROOT}"/dev/sd[a-d][a-z]* "${ROOT}"/dev/sd[e-z]* # excess sata | scsi
+               rm -f "${ROOT}"/dev/tty[a-z]* "${ROOT}"/dev/tty3[2-9] "${ROOT}"/dev/tty[4-6]? # excess tty
        fi
 }
Comment 32 William Hubbs gentoo-dev 2011-05-29 23:03:57 UTC
Created attachment 275049 [details]
devices.patch

Mike,

this patch is based on Jorge's patch, but I had to add one more
expression to the tty removals to get my base set down; I had over 3000
nodes after Jorge's patch.

I added the following expression to the tty removals:
"${ROOT}"/dev/tty[A-Z]*

Does this create an acceptable base set of devices?
Comment 33 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2011-05-29 23:17:58 UTC
+1 from me.
Comment 34 SpanKY gentoo-dev 2011-05-30 02:36:41 UTC
odd, i'd of thought "console" would be part of "std" (standard).  that also reminds me to make sure we have ptmx.

i left tty{0..63} alone because that is how many the kernel compiles in with the virtual console.  there is no way to configure this to be differently without modifying the source code.  simply look in /sys/class/tty/ to see them.  if we're going to kill the high tty's, then might as well do 20+.

i imagine the difference in tty counts is locale.  mine does case-insensitive matching (en_US).

http://sources.gentoo.org/sys-apps/makedev/makedev-3.23.1.ebuild?r1=1.7&r2=1.8
Comment 35 William Hubbs gentoo-dev 2011-05-30 16:05:04 UTC
(In reply to comment #34)
> i left tty{0..63} alone because that is how many the kernel compiles in with
> the virtual console.  there is no way to configure this to be differently
> without modifying the source code.  simply look in /sys/class/tty/ to see them.

In that case, I'm fine with tty0..63 being there; we probably should not kill these.
Comment 36 Alex Legler (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2011-06-03 11:17:01 UTC
*** Bug 369873 has been marked as a duplicate of this bug. ***
Comment 37 Jeroen Roovers gentoo-dev 2011-06-10 02:27:45 UTC
*** Bug 370733 has been marked as a duplicate of this bug. ***
Comment 38 Gert Andersen 2011-06-10 14:14:34 UTC
Hi.

I have tried this install first on a x86 and then on a amd K8 X2 box mashine and on both allway used genkernel all on both computers.
On the x86 pc with minimal install and on the AMD K8 X2 computer.

On the X86 pac mashine was there no messages of error by rn genkernel all but on the AMD K8 X2 pc, come genkernel all with error compiling modules.

When I download the latest stage3-*.tar.bz2 and portage-latest.tar.bz2 files for both system by X86 i686 and x86_64 (amd64) downloads og stage3 and portage file, and with amd64 minimal install is genkernel all come with the error compile modules for about 10-30 or more modules.

So is the latest stage3 and portage files been updated for the missing things for it is done well ?

The first try of install was at 5 June 2011 on the x86 and the last try on the amd k8 X2 was 8 June.

The handbooks shoudl be update by the version of kernel as it still use the 2.6.34 kernel as installation.
Comment 39 Gert Andersen 2011-06-10 14:17:04 UTC
Hi.

I have tried this install first on a x86 and then on a amd K8 X2 box mashine and on both allway used genkernel all on both computers.
On the x86 pc with minimal install and on the AMD K8 X2 computer.

On the X86 pac mashine was there no messages of error by rn genkernel all but on the AMD K8 X2 pc, come genkernel all with error compiling modules.

When I download the latest stage3-*.tar.bz2 and portage-latest.tar.bz2 files for both system by X86 i686 and x86_64 (amd64) downloads og stage3 and portage file, and with amd64 minimal install is genkernel all come with the error compile modules for about 10-30 or more modules.

So is the latest stage3 and portage files been updated for the missing things for it is done well ?

The first try of install was at 5 June 2011 on the x86 and the last try on the amd k8 X2 was 8 June.

The handbooks shoudl be update by the version of kernel as it still use the 2.6.34 kernel as installation.
Comment 40 Chí-Thanh Christopher Nguyễn gentoo-dev 2011-06-13 20:38:02 UTC
*** Bug 371263 has been marked as a duplicate of this bug. ***
Comment 41 Jeroen Roovers gentoo-dev 2011-06-15 16:22:03 UTC
*** Bug 371787 has been marked as a duplicate of this bug. ***
Comment 42 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-06-19 14:00:57 UTC
Can we please get the following applied to baselayout-2.0.3 and work on getting it marked stable so we can get our devices back?
If base-system gives me the ok I can commit it.

diff -u -b -B -r1.1 baselayout-2.0.3.ebuild
--- baselayout-2.0.3.ebuild     18 Jun 2011 04:02:34 -0000      1.1
+++ baselayout-2.0.3.ebuild     19 Jun 2011 13:59:43 -0000
@@ -14,7 +14,10 @@
 KEYWORDS="~alpha ~amd64 ~arm ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~x86 ~sparc-fbsd ~x86-fbsd"
 IUSE="build"
 
-PDEPEND="sys-apps/openrc"
+PDEPEND="
+       sys-apps/makedev
+       sys-apps/openrc
+"
 
 pkg_preinst() {
        # Bug #217848 - Since the remap_dns_vars() called by pkg_preinst() of
Comment 43 William Hubbs gentoo-dev 2011-06-19 15:49:09 UTC
(In reply to comment #42)
> Can we please get the following applied to baselayout-2.0.3 and work on getting
> it marked stable so we can get our devices back?
> If base-system gives me the ok I can commit it.

I'm not sure whether we should do this by adding makedev to pdepend as suggested here or by adding makedev to the default/linux profile as part of the system set.
Comment 44 SpanKY gentoo-dev 2011-06-19 17:48:09 UTC
that style is broken, so i committed it myself

http://sources.gentoo.org/sys-apps/baselayout/baselayout-2.0.3.ebuild?r1=1.1&r2=1.2
Comment 45 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-06-19 17:53:32 UTC
(In reply to comment #44)
> that style is broken, so i committed it myself
> 
> http://sources.gentoo.org/sys-apps/baselayout/baselayout-2.0.3.ebuild?r1=1.1&r2=1.2

Thanks.

PS - viewvc fails to show the diff for most diff styles. I'll poke infra about it.
PPS - I understand you might not like it, but the style isn't "broken" - it's currently being used on the tree.
Comment 46 SpanKY gentoo-dev 2011-06-19 18:13:58 UTC
your logic is invalid.  people doing X doesnt mean X is not broken.
Comment 47 William Hubbs gentoo-dev 2011-06-20 04:37:18 UTC
@releng, @base-system:

We need to reconsider how we are approaching this bug.

Thanks to Nirbheek assisting me with researching, we found that Debian, fedora and ubuntu all have at least CONFIG_DEVTMPFS set to y in their kernels and have done this for quite some time

If we do this and set CONFIG_DEVTMPFS_MOUNT to y as well, that would mount devtmpfs on /dev and the base devices for the system would be generated automatically on bootup.

We could then make sure that these options are set in the default configs for  genkernel and get udev to add the symlink to make sure it is in the sysinit runlevel for the stages.

The final step would be to have the docs team document that if someone compiles their own kernel they would need CONFIG_DEVTMPFS and CONFIG_DEVTMPFS_MOUNT set to y.

If we do that, it will eliminate the need for makedev entirely and give us dynamic devices instead of creating all of these static device nodes.

I do not understand what the objection to this setup would be.
Comment 48 Nirbheek Chauhan (RETIRED) gentoo-dev 2011-06-20 05:39:38 UTC
To add to what William said, Ubuntu and Fedora also enable DEVTMPFS_MOUNT by default in their kernel sources, and we should do the same for genkernel + gentoo-sources. This way, it won't be a documentation regression either. Only people who use vanilla-sources will have to know about it.

About makedev, as I understand it, it'll create device nodes on the filesystem, and those nodes will be used when /dev isn't mounted. The old use-cases for this were:

(a) Embedded devices where udev isn't used at all because the hardware is static.
(b) Recovery mode when udev fails to start, so you can fix the problem.
(c) Embedded devices which either use an old kernel, or patch the kernel to disable hardware probing.

CONFIG_DEVTMPFS and CONFIG_DEVTMPFS_MOUNT mean that the kernel does hardware probing, creates the corresponding device nodes, and mounts /dev. Hence, there's no userspace dependency for getting a bootable system. This covers cases (a) and (b).

Case (c) is not covered by devtmpfs, and is not covered by the standard makedev list shipped by us anyway, since the device will want custom nodes for its own purpose.

The only case I can imagine where you'd want something in /dev is if the user accidentally disabled DEVTMPFS_MOUNT, and udev failed to start as well. For this case, you just need /dev/console and /dev/null on the root filesystem.

Infact, I'd argue that a fallback to a mostly-populated static /dev is harder to notice for the user than an almost-empty /dev that just barely allows boot.

Finally, pretty much no other distro uses makedev anymore. If we continue using it, we'll diverge from everyone else, and increase our long-term workload.
Comment 49 William Hubbs gentoo-dev 2011-06-20 21:45:35 UTC
All,

where we are now basically is this:

If we want to use makedev, we have to either stabilize it on all arch's
ourselves or wait for it to be stabilized. The stable request is the
dependency for this bug.

Once that is done, we can add makedev to the dependencies for
baselayout-2 for kernel_linux users when USE=build to make sure
makedev gets pulled into
stage 1.  

On the other hand, if we take the dynamic approach, we
need to make sure that CONFIG_DEVTMPFS and CONFIG_DEVTMPFS_MOUNT are set
to y in the default kernel configurations for genkernel and possibly the
specs for the minimal cds as well.

Either way, we still need to make sure that udev gets added to the
sysinit runlevel in the stages.
Comment 50 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2011-06-20 21:48:56 UTC
If the users boots a kernel that does not:
1. Have CONFIG_DEVTMPFS set
1.1. This case I don't know
2. Have CONFIG_DEVTMPFS_MOUNT set (but does have the base CONFIG_DEVTMPFS available).
2.1. We can mount it ourselves.

I say we should take both the static & dynamic approaches, and have it run makedev if DEVTMPFS is not available.
Comment 51 William Hubbs gentoo-dev 2011-06-20 23:02:35 UTC
(In reply to comment #50)
> If the users boots a kernel that does not:
> 1. Have CONFIG_DEVTMPFS set
> 1.1. This case I don't know

I will test this case in a little while and report back.

> 2. Have CONFIG_DEVTMPFS_MOUNT set (but does have the base CONFIG_DEVTMPFS
> available).
> 2.1. We can mount it ourselves.

My test of the first case above will probably also affect what we do here. I'm thinking that if we do not have /dev populated it will just slow down the boot process slightly because  udev will have to populate it, so there may not be anything we have to do.

> I say we should take both the static & dynamic approaches, and have it run
> makedev if DEVTMPFS is not available.

Off the top of my head, what I'm thinking will happen is that if devtmpfs and devtmpfs_mount are not available, we just need /dev/console and /dev/null in the stages. If udev is in the sysinit runlevel in the stages, my understanding is that it will populate /dev based on the user's system. Getting it there is covered in bug #369037.

If the user doesn't want to use udev, they can use sys-fs/static-dev which would give them a statically managed /dev.
Comment 52 William Hubbs gentoo-dev 2011-06-21 01:39:58 UTC
(In reply to comment #50)
> If the users boots a kernel that does not:
> 1. Have CONFIG_DEVTMPFS set
> 1.1. This case I don't know
> 2. Have CONFIG_DEVTMPFS_MOUNT set (but does have the base CONFIG_DEVTMPFS
> available).
> 2.1. We can mount it ourselves.

This is all handled by the udev-mount service script, which is needed by the udev service script.

Once udev is successfully started, /dev is mounted from fstab, a devtmpfs, or a tmpfs, so whatever is stored in the /dev directory on the drive is moot.

Once we solve bug #369037, the only time a user will be affected by the devices in the stages is if the user decides to remove udev from sysinit or if starting udev fails.

So, the only question left on this bug for me is, how many devices do we want in the stages if a user selects to not run udev or if udev fails?
Comment 53 SpanKY gentoo-dev 2011-06-21 03:34:56 UTC
sorry, but no.  making CONFIG_DEVTMPFS a requirement for kernels is unacceptable.  fedora/ubuntu/etc... also make it fairly hard to boot a system without a built in initramfs that bootstraps things.  you want to go that route too ?

Gentoo works great without all this crap filling up the boot setup.  you can take any vanilla kernel and have it just work.  ive never built a kernel with genkernel, nor do i plan on starting now, and i imagine a lot of people out there do as well.  the system we have now is trivial to maintain and fairly robust.  and you want to throw it out for no good reason ?  i'm sure people will love not seeing their system boot because a new config option is now mandatory.

as for us "diverging", i guess you havent booted Debian/Ubunutu in a while.  both install makedev by default.  and they ship with a crap ton of device nodes generated in their root.  the package itself is trivially small.
Comment 54 SpanKY gentoo-dev 2011-06-21 03:50:41 UTC
Jorge: could you test stage building with baselayout having:
PDEPEND="build? ( sys-apps/makedev )"

we really only need it for that step, and it makes it easy for people to prune it from their system later on.
Comment 55 SpanKY gentoo-dev 2011-06-21 04:06:31 UTC
another data point: DEVTMPFS is only a recent feature.  other distros have the luxury of forcing recent kernel versions and punting everyone else.  we still work great on old kernels, and i dont see this as sufficient justification to say to those people "too bad".  i know people run old stuff because when things break on them, we get reports semi-quickly.

even udev isnt a hard requirement.  it just makes live a lot easier for people.
Comment 56 William Hubbs gentoo-dev 2011-06-21 06:00:58 UTC
(In reply to comment #53)
> sorry, but no.  making CONFIG_DEVTMPFS a requirement for kernels is
> unacceptable.  fedora/ubuntu/etc... also make it fairly hard to boot a system
> without a built in initramfs that bootstraps things.  you want to go that route
> too ?

I could have probably made my position a little more clear in comment 52. Because of the way the udev/udev-mount scripts work, devtmpfs is not necessary, so I will agree that we shouldn't require it.

(In reply to comment #54)
> Jorge: could you test stage building with baselayout having:
> PDEPEND="build? ( sys-apps/makedev )"
> we really only need it for that step, and it makes it easy for people to prune
> it from their system later on.

Jorge, if this works, I agree that we should give people the option to prune makedev from their systems.

I have another question for you as well. If this does work, I think we can drop makedev from PDEPEND for baselayout and just add makedev as a separate package to merge in stage 1 the same way we are merging udev. What do you think?
Comment 57 SpanKY gentoo-dev 2011-06-21 15:52:14 UTC
that's fine by me
Comment 58 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-06-21 18:03:26 UTC
(In reply to comment #56)
> (In reply to comment #53)
> (In reply to comment #54)
> > Jorge: could you test stage building with baselayout having:
> > PDEPEND="build? ( sys-apps/makedev )"
> > we really only need it for that step, and it makes it easy for people to prune
> > it from their system later on.
> 
> Jorge, if this works, I agree that we should give people the option to prune
> makedev from their systems.
> 
> I have another question for you as well. If this does work, I think we can drop
> makedev from PDEPEND for baselayout and just add makedev as a separate package
> to merge in stage 1 the same way we are merging udev. What do you think?

Adding custom code to catalyst is something releng really doesn't like and wants to avoid. So in this case I think we should follow Mike's proposal. I don't see why there should be any problem with it, but I'm going to test it in any case.
Comment 59 SpanKY gentoo-dev 2011-06-21 18:13:19 UTC
you dont need to touch catalyst.  i think William is referring to scripts/bootstrap.sh in the portage tree.
Comment 60 William Hubbs gentoo-dev 2011-06-21 18:57:40 UTC
The code that picks the packages to put in stage 1 is internal to
catalyst, so that is what I was referring to. Basically it does an
"emerge --nodeps [list of packages]" to create stage 1.
Comment 61 SpanKY gentoo-dev 2011-06-21 19:03:33 UTC
does it actually hardcode a list ?  what's the point of packages.build in the profile then ?
Comment 62 William Hubbs gentoo-dev 2011-06-21 20:31:12 UTC
No, it doesn't hard code a list; it appears to use the packages.build
files.

Jorge, if I understand things correctly, why can't we just update the
profiles/default/linux/packages.build file to include udev and makedev?
Comment 63 SpanKY gentoo-dev 2011-06-21 21:03:38 UTC
why udev ?  we dont need that until stage3, and it should be part of system ...
Comment 64 William Hubbs gentoo-dev 2011-06-21 22:17:59 UTC
Udev needs to be added to the sysinit runlevel and
udev-postmount needs to be added to the default runlevel in the stages.

If we don't do this, udev will not start when a user boots a newly
installed system.

Starting udev was handled by addons in baselayout-1, but openrc treats
it like any other service since it doesn't support addons.

The choices I know of for making this happen are to pull udev into
stage1 so that we can have postinst check for USE=build or have users
execute the rc-update commands to add these while they are in the chroot
before they reboot their systems.

Do you know of a third choice?
Comment 65 Jeremy Olexa (darkside) (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2011-06-21 22:57:48 UTC
(In reply to comment #64)

> Do you know of a third choice?

The udev ebuild can add it to the default runlevel in postinst() ?
Comment 66 William Hubbs gentoo-dev 2011-06-21 23:42:58 UTC
(In reply to comment #65)
> (In reply to comment #64)
> 
> > Do you know of a third choice?
> 
> The udev ebuild can add it to the default runlevel in postinst() ?

That is what is happening right now, but only if  build is in the use flags, which is how we indicate that stage 1 is being built.

I don't want to add udev and udev-postmount to runlevels unconditionally because that gets us into adding services on users' systems.
Comment 67 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-06-22 02:04:49 UTC
(In reply to comment #66)
> (In reply to comment #65)
> > (In reply to comment #64)
> > 
> > > Do you know of a third choice?
> > 
> > The udev ebuild can add it to the default runlevel in postinst() ?
> 
> That is what is happening right now, but only if  build is in the use flags,
> which is how we indicate that stage 1 is being built.
> 
> I don't want to add udev and udev-postmount to runlevels unconditionally
> because that gets us into adding services on users' systems.

I agree with William on this one. If udev were to install the symlink unconditionally, we would be "stepping" into the user's domain.
Comment 68 Thomas Beierlein gentoo-dev 2011-06-22 06:46:17 UTC
*** Bug 372535 has been marked as a duplicate of this bug. ***
Comment 69 SpanKY gentoo-dev 2011-06-22 17:10:26 UTC
udev is missing IUSE=build, so i didnt see the stage1 merge making sense

along those lines, i dont see how the `use build` logic can kick in ... i thought portage only allowed `use` to work against things in $IUSE ...
Comment 70 William Hubbs gentoo-dev 2011-06-22 19:48:32 UTC
(In reply to comment #69)
> udev is missing IUSE=build, so i didnt see the stage1 merge making sense

Good catch, this is now fixed.

> along those lines, i dont see how the `use build` logic can kick in ... i
> thought portage only allowed `use` to work against things in $IUSE ...

I'm honestly not sure what portage does for use flags not listed in IUSE. I do know that I didn't catch this and repoman either didn't complain or didn't make a big deal of it. That is probably a topic for a different bug.
Comment 71 Kyle Sanderson 2011-06-26 08:37:12 UTC
Any update on a new Stage3 with this fixed? I nearly reformatted (thinking I was at fault) before someone showed me this bug. If not, is there a current ISO that doesn't have this critical issue?
Comment 72 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-06-26 15:28:25 UTC
(In reply to comment #71)
> Any update on a new Stage3 with this fixed? I nearly reformatted (thinking I
> was at fault) before someone showed me this bug. If not, is there a current ISO
> that doesn't have this critical issue?

Your issue is with the stages, not the ISO images. We're working on fixing the tree to get working stages.

If anyone is interested on this, please test the stages[1] I've built locally with the required changes to the tree.

 [1] - http://www.jmbsvicetto.name/releases/
Comment 73 Daniel Beckham 2011-06-26 16:39:32 UTC
Since fixing this is fairly low priority, can someone please detail what actual changes I need to make to my chrooted installation to get this working properly? It seems that turning on the CONFIG_DEVTMPFS and CONFIG_DEVTMPFS_MOUNT may somewhat fix this, but I'd rather do it the right way instead of having custom kernel settings that have to be remember for the life of this install.

I have several systems that are waiting to go into production and need an OS installed. Even a URL or where to actually find a working pre 5/26 stage3 would be awesome.
Comment 74 Mike Nerone 2011-06-26 20:17:18 UTC
(In reply to comment #73)
> Since fixing this is fairly low priority, can someone please detail what actual
> changes I need to make to my chrooted installation to get this working
> properly?

######## SUMMARY OF HOW TO FIX THIS MANUALLY #########

I was struck by the bug, and the following is all I had to do (note: you don't even have to chroot):

1. After booting a live CD and mounting the installed image at /mnt/gentoo, create the necessary device nodes:

    cd /mnt/gentoo/dev
    rm -f null # The existing file is wrong.
    mknod --mode=600 console c 5 1
    mknod --mode=666 null c 1 3

2. Add udev to the sysinit runlevel

    ln -sfn /etc/init.d/udev /mnt/gentoo/etc/runlevels/sysinit/
    # OR you could chroot and "rc-update add udev sysinit" if you like.

I hope that helps some people.

@ Gentoo Devs
This bug was very frustrating, causing me perhaps 12 hours of hair-pulling spread over a couple of weeks, rebuilding my kernel many times trying to figure out what I was doing wrong, which turned out to be absolutely nothing. If fixing the build scripts is going to take a while, perhaps as a stopgap, someone could manually fix the dev nodes in the stages for now (and either enable udev there, as well, or add a trivial paragraph to the guide about enabling it)? It would take 15-20 minutes of labor to fix the stages and save potentially a whole lot of people from wasting their time, becoming frustrated, and for first-timers, getting a very bad impression of Gentoo. Please?

Incidentally, this bug report was not particularly helpful in determining a treatment, because the summary contains only the diagnosis, while affected users have only the symptom for the purpose of formulating their searches. Someone will probably point out some search incantation that to their own eyes is "obvious", but it will likely only be obvious in retrospect - I know I didn't find it (and eventually only found the bug because of a forum post, which itself was only marginally easier to find).
Comment 75 Mike Nerone 2011-06-26 20:19:15 UTC
BTW, notable things I did *NOT* have to do to fix it:

* Enable anything DEVTMPFS related in the kernel.
* Create the /dev/zero device node, or any other nodes that have been mentioned, except for /dev/console and /dev/null.
Comment 76 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-06-27 00:05:46 UTC
(In reply to comment #73)
> Since fixing this is fairly low priority, can someone please detail what actual
> changes I need to make to my chrooted installation to get this working
> properly? It seems that turning on the CONFIG_DEVTMPFS and
> CONFIG_DEVTMPFS_MOUNT may somewhat fix this, but I'd rather do it the right way
> instead of having custom kernel settings that have to be remember for the life
> of this install.
> 
> I have several systems that are waiting to go into production and need an OS
> installed. Even a URL or where to actually find a working pre 5/26 stage3 would
> be awesome.

The solution we're going to take to fix this will include the following:
 * add sys-apps/makedev and sys-fs/udev to $(portageq portdir)/profiles/default/linux/packages.build which will create the devices on /dev and add udev to the sysinit runlevel on stage1
 * enable CONFIG_DEVTMPFFS* for genkernel and ask docs team to document it on the installation handbook

In any case, you can use the stages I've built until we get new official stages - they include the first point above.
Comment 77 Kyle Sanderson 2011-06-27 01:14:16 UTC
Thanks Mike!

I would have been absolutely lost without those instructions. Hopefully this is resolved in an official capacity soon.
Comment 78 Mike Nerone 2011-06-27 01:32:52 UTC
(In reply to comment #76)
> In any case, you can use the stages I've built until we get new official stages
> - they include the first point above.

Jorge,

All I'm is that by the time somebody finds your stages, it means they've already found this bug, which means they've probably already put several hours of work into the problem. Certainly something better than that can be done, right? Of course, if we're saying new stages will be ready in a day or two, then I'm with ya.

Mike Nerone
Comment 79 SpanKY gentoo-dev 2011-06-27 03:04:30 UTC
the issue is in the process of being fixed.  posting long rants that literally add nothing to move things along isnt terribly useful.
Comment 80 Mike Nerone 2011-06-27 03:25:29 UTC
Hahaha - if you think that was a rant, you don't know me at all.

After I went to the effort to try to help the people with a problem, you really *should* have been a little more gracious and just said, "I do understand where you're coming from, but I think it's best if we just stick with the current plan. Thanks for the input, though."

No need to add insult to injury.
Comment 81 SpanKY gentoo-dev 2011-06-27 03:32:46 UTC
i did not knock you for assisting anyone.

have you added anything to the current plan, or any viable alternatives ?  i havent seen any suggestions.
Comment 82 Mike Nerone 2011-06-27 04:22:28 UTC
(In reply to comment #81)
> i did not knock you for assisting anyone.
> 
> have you added anything to the current plan, or any viable alternatives ?  i
> havent seen any suggestions.

Then perhaps you should read that "rant" you referred to in the first place from comment #74 again, which was actually a courteous suggestion to fix the stages manually to help those who don't know about this bug that's persisted for over a month now (though I was too polite to point that out - I see now that that was misguided).

Awaiting your next unnecessary dig.
Comment 83 Kerin Millar 2011-06-27 05:44:22 UTC
> saying "you must always run udev" to get a sane system isnt acceptable.  not
> only has that shown many times here to result in a flaky system, but it's shown
> in the past as well.  so simply creating the bare min doesnt make sense.  we
> want a sane base set sitting on the rootfs.

I'm somewhat late to the party but, for what it's worth, I couldn't agree more. Retaining a set of base device nodes is the sensible thing to do. Arguably, udev has a track record of being the single most volatile component in @system, not least due to its fiddly prerequisites and the high price of its failure. Ergo, the version of the kernel can matter; a particular combination of Kconfig options must be in effect; it still needs _some_ device nodes to  exist beforehand; unexpected/breaking changes may be propagated from upstream (bug 207612 being a memorable example). That it fails so miserably in this particular case - with no obvious indicators as to the nature of the problem - is surprising and perturbing for new users and old hands alike.

Whittling down the static device nodes to the bare 'essentials' would gain absolutely nothing. Instead, it would make recovery situations harder and, ultimately, increase the probability of a Gentoo system not being able to boot at some point in its lifespan. Particularly for those who use Gentoo in production, the stock device nodes provide some measure of assurance that things are going to stay working or that things will at least be easier to fix should they break. That goes especially for remote/headless systems.

From what I can see, Jorge seems to be aiming for a resolution that will make things much as they were (in which case I, for one, will be delighted). Still, SpanKY's comment struck a chord ... let's not ever be seduced by the notion of a minimalist rootfs /dev for the reasons stated above.
Comment 84 Kyle Sanderson 2011-06-27 06:43:31 UTC
I have to agree, this is putting a band aid on a huge issue. If udev fails to start, the entire system has the potential to halt. This is a disservice to headless systems, newbies to Gentoo (like myself), and everyone else who's still learning their way around Linux and Gentoo. Having the device nodes exist on the rootfs (and not be created on the fly) is really the only plausible solution to fixing this and not have it regress later.
Comment 85 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-06-27 10:47:49 UTC
(In reply to comment #84)
> I have to agree, this is putting a band aid on a huge issue. If udev fails to
> start, the entire system has the potential to halt. This is a disservice to
> headless systems, newbies to Gentoo (like myself), and everyone else who's
> still learning their way around Linux and Gentoo. Having the device nodes exist
> on the rootfs (and not be created on the fly) is really the only plausible
> solution to fixing this and not have it regress later.

Please read comment 76 again.

I'm also going to propose the Docs team adds a note in the installation handbook, kernel section, about we recommending users to enable the CONFIG_DEVTMPFS and CONFIG_DEVTMPS_MOUNT so they get the devices for their hardware created by the kernel before passing control to userland and udev kicks in. This also helps users that don't want to use udev and might help if users have a problem with udev.
Comment 86 Kerin Millar 2011-06-27 19:46:03 UTC
I've just looked at exactly how the sys-apps/makedev approach works, as per Comment 76. Nice work. It will keep those of us who prefer the old-school approach happy. Jorge, what degree of testing is required on your stages before they're considered fit for pushing out to the official mirrors? Is there anything we punters can do to help move this along?
Comment 87 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-06-27 20:07:41 UTC
(In reply to comment #86)
> I've just looked at exactly how the sys-apps/makedev approach works, as per
> Comment 76. Nice work. It will keep those of us who prefer the old-school
> approach happy. Jorge, what degree of testing is required on your stages before
> they're considered fit for pushing out to the official mirrors? Is there
> anything we punters can do to help move this along?

Kerin,

I unpacked them here and looked at /dev to check for the devices and at /etc/runlevels/sysinit to ensure the udev runlevel was there and wasn't broken.
I'll push the changes to packages.build tonight and ask William to push the changes to udev asap.

Mike,

going this route we can drop makedev from baselayout dependencies. Can you do it? Sorry for the extra work, but packages.build seems a better route.
Comment 88 SpanKY gentoo-dev 2011-06-27 21:14:15 UTC
(In reply to comment #82)

saying "please fix the stages" is useless noise.  so the answer to my previous implicit question is "no, nothing useful has been suggested".  thanks for clarifying.
Comment 89 SpanKY gentoo-dev 2011-06-27 21:15:11 UTC
(In reply to comment #87)

just lemme know when
Comment 90 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-06-28 01:55:56 UTC
(In reply to comment #89)
> (In reply to comment #87)
> 
> just lemme know when

http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/profiles/default/linux/packages.build?r1=1.4&r2=1.5

Done, please do it as soon as you can.
Comment 91 Mike Nerone 2011-06-28 16:24:52 UTC
(In reply to comment #88)
> saying "please fix the stages" is useless noise.  so the answer to my previous
> implicit question is "no, nothing useful has been suggested".  thanks for
> clarifying.

Aww, I'm sad. I thought we were done with this. I'll tango as long as you insist. I didn't just say "fix the stages". I specifically suggested the *new*, admittedly interim, solution of putting *manually* fixed stages at the official download locations while the admirable effort of fixing the automated build is completed. It doesn't bother me if you're against the idea - that's what discussion is for for *most* of us. But that doesn't make it "useless" to consider it. The only thing truly "useless" is your rudeness and all the crap that followed it.

If you enjoy berating others and feel that doing so actually merits defending, by all means respond again.
Comment 92 Gert Andersen 2011-06-28 17:48:35 UTC
Hi.

All ways on fixing the modules problem compiling is not working on my system self by use the commands to make the /dev/null and /dev/console, and genkernel is sure not do it either.
emerge genkernel 
genkernel all
compile the image file
compile modules
results is error compile them and no initramfs is made, only the kernel is there.
Then is there the good file and program Midnight Commander (MC) to get installed
emerge app-misc/mc
it make gentoo to look 12 files and it selv (13 files) and compile them all.
Then ups what is there now been done, about 10 modules + some libs is been compiled.
The last compiled source is mc self.
Then try now to run genkernel all 
results is no error compiling modules and initramfs is compiled too and is done.

Go on to the other gentoo work and come to grub and the 2 last thing before reboot to harddisk boot up.
emerge grub and look what it said and setup grub.conf for work.
install grub in to mbr on hsrddkisk.

Then is there now bug again and it is grub there have a bug with a file in /boot/grub/
try run grub-install --no-floppy /dev/sda
error comes with and --no-floppy isn't work as it shall
this first give i/o error looking for a fd0 there not is there
edit device.map and remove (fd0) 
try the same command again
bug error with file stage1
try install grub manual
grub --no-floppy
root (hd0,0)
setup (hd0)
bingo bug error
quit
setup (hd0) and use the default way gives error or bug error by stage1 file in /boot/grub/
This is so grub not is installed to mbr and is working.

Fix this too ASAP!!

The best way to that kernel 2.6.38 can be made and work is to install a older kernel like 2.6.34/35 first and then updaate kernel and initramfs by a running system.
Comment 93 Kerin Millar 2011-06-28 21:41:33 UTC
(In reply to comment #92)
> All ways on fixing the modules problem compiling is not working on my
> system self by use the commands to make the /dev/null and
> /dev/console, and genkernel is sure not do it either.

The instructions for the workaround mentioned earlier are correct, but this bug has nothing to do with a "modules problem compiling". This bug is for the discussion - and resolution - of the problem of missing device nodes in the stage3 tarball only. If the device nodes are missing, then Gentoo may unexpectedly freeze during the boot process.

> results is error compile them and no initramfs is made, only the kernel
> is there.

If you are certain that there is a bug in genkernel regarding initramfs image generation, please file a separate bug. It's not relevant here.

> emerge app-misc/mc
> it make gentoo to look 12 files and it selv (13 files) and compile
> them all.

You can learn about how to take advantage of USE flags if you want to better control the package dependency graph. But, again, it's not relevant here.

> This is so grub not is installed to mbr and is working.

I'm sorry you had difficulties in finishing your installation, especially if you are a new user. However, please understand that Bugzilla is not a general support forum. I would encourage you to visit the #gentoo or #gentoo-dk channel at irc.freenode.net, or any of the other avenues of support such as forums.gentoo.org and the mailing lists. There should be plenty of helpful people there who will be eager to help you resolve these problems.
Comment 94 Andrey Vihrov 2011-06-29 06:55:33 UTC
Hello to the party,

What do people who installed Gentoo in the mean time fixing this manually (like in [1]) have to do now to keep consistency? Do we merge makedev with /dev unmounted?

[1] http://dev.gentoo.org/~a3li/openrc.txt
Comment 95 Gert Andersen 2011-06-29 08:07:18 UTC
(In reply to comment #93)
> (In reply to comment #92)
> > All ways on fixing the modules problem compiling is not working on my
> > system self by use the commands to make the /dev/null and
> > /dev/console, and genkernel is sure not do it either.
> The instructions for the workaround mentioned earlier are correct, but this bug
> has nothing to do with a "modules problem compiling". This bug is for the
> discussion - and resolution - of the problem of missing device nodes in the
> stage3 tarball only. If the device nodes are missing, then Gentoo may
> unexpectedly freeze during the boot process.

I though the disussion was on the missing modules too, as I first have it by using genkernel and had it to not work and had use the workaround way from comment 74. But I should maybe have used it from another earlier comment to work better. 

> > results is error compile them and no initramfs is made, only the kernel
> > is there.
> If you are certain that there is a bug in genkernel regarding initramfs image
> generation, please file a separate bug. It's not relevant here.

It is about the missing modules by if genkernel not find the modules is gives error compile the modules, and the initramfs is not made.
By emerge mc the midnight commander make that the missing modules got ind and then do genkernel compile the modulesn and make initramfs finished.
> > emerge app-misc/mc
> > it make gentoo to look 12 files and it selv (13 files) and compile
> > them all.
> You can learn about how to take advantage of USE flags if you want to better
> control the package dependency graph. But, again, it's not relevant here.

The handbook should be updated better as it use a old kernel as 2.6.34 and it is old and not showig the new things with kernel 2.6.38

> > This is so grub not is installed to mbr and is working.
> I'm sorry you had difficulties in finishing your installation, especially if
> you are a new user. However, please understand that Bugzilla is not a general
> support forum. I would encourage you to visit the #gentoo or #gentoo-dk channel
> at irc.freenode.net, or any of the other avenues of support such as
> forums.gentoo.org and the mailing lists. There should be plenty of helpful
> people there who will be eager to help you resolve these problems.
The grub install is too a problem after the other thing is worked around, but could be it should post as a bug for grub.
Comment 96 Mark Zhitomirski 2011-06-29 11:52:22 UTC
(In reply to comment #94)
> What do people who installed Gentoo in the mean time fixing this manually (like
> in [1]) have to do now to keep consistency? Do we merge makedev with /dev
> unmounted?
> 
> [1] http://dev.gentoo.org/~a3li/openrc.txt

aside from noticing documentation issue too i'm at loss with the fact that neither openrc/baselayout nor udev consider /dev/console and freinds belonging to them.
Upon stumbling this bug (being unaware of it, shame on me) i've tried to emerge everything in question (udev, openrc, system, -e world) to no success. Should not this issue be refered to udev/baselayout in the first place?
Comment 97 Kerin Millar 2011-06-29 17:23:41 UTC
(In reply to comment #94)
> What do people who installed Gentoo in the mean time fixing this manually
> (like in [1]) have to do now to keep consistency? Do we merge makedev with
> /dev unmounted?

To achieve parity with an installation that was based upon a correctly built stage tarball:-

Method 1
--------
# emerge -n1 makedev
# mkdir /mnt/rootfs
# mount -o bind / /mnt/rootfs
# cd /usr/portage/sys-apps/makedev
# USE="build" ROOT="/mnt/rootfs" ebuild makedev-3.23.1.ebuild postinst
# umount /mnt/rootfs

Method 2
--------
Alternatively, boot from a LiveCD, chroot (but do NOT bind mount /dev to /mnt/gentoo/dev) then run:-

# USE="build" emerge -1 makedev

The second method is also the simplest way to fix the problem on a new installation i.e. as an alternative to a3li's instructions.
Comment 98 Andrey Vihrov 2011-06-29 17:52:37 UTC
(In reply to comment #97)
> # mount -o bind / /mnt/rootfs

That's one clever solution!
Comment 99 Kerin Millar 2011-06-29 17:54:54 UTC
(In reply to comment #96)
However the device nodes are installed, it has always occurred in the pkg_postinst() phase during stage building so, strictly speaking, they do not "belong" to the package. Before openrc (which requires baselayout-2) was stabilised, baselayout-1 was the default. You can see in its ebuild that the device nodes are created where USE="build" is in effect.

I don't speak for the developers, but it seems that this was entirely overlooked in the transition to baselayout-2, hence this bug. Now baselayout is a 'skeleton' package and of diminished importance. Yes, you could argue that the responsibility still belongs there but, as per the developer commentary above, the decision has *already* been made to shift the process to sys-apps/makedev and add it to packages.build, which is a sufficiently elegant solution. It will ensure that there is a full complement of essential device nodes in the stage tarballs, as was the case before. The problem remains that the tarballs at the official mirrors have yet to benefit.
Comment 100 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-06-29 18:58:09 UTC
Kerin,

I've already upload my amd64 stages to the mirrors, so the 20110625 stages on http://distfiles.gentoo.org/releases/amd64/autobuilds/current-stage3/ have the devices back and the udev symlink.

(In reply to comment #97)
> (In reply to comment #94)
> > What do people who installed Gentoo in the mean time fixing this manually
> > (like in [1]) have to do now to keep consistency? Do we merge makedev with
> > /dev unmounted?
> 
> To achieve parity with an installation that was based upon a correctly built
> stage tarball:-
> 
> Method 1
> --------
> # emerge -n1 makedev
> # mkdir /mnt/rootfs
> # mount -o bind / /mnt/rootfs
> # cd /usr/portage/sys-apps/makedev
> # USE="build" ROOT="/mnt/rootfs" ebuild makedev-3.23.1.ebuild postinst
> # umount /mnt/rootfs
> 
> Method 2
> --------
> Alternatively, boot from a LiveCD, chroot (but do NOT bind mount /dev to
> /mnt/gentoo/dev) then run:-
> 
> # USE="build" emerge -1 makedev
> 
> The second method is also the simplest way to fix the problem on a new
> installation i.e. as an alternative to a3li's instructions.

I suggest using method 2.
Thanks for presenting these alternatives for users with "broken" stages.
Comment 101 Naohiro Aota gentoo-dev 2011-07-02 16:55:13 UTC
(In reply to comment #44)
> that style is broken, so i committed it myself
> 
> http://sources.gentoo.org/sys-apps/baselayout/baselayout-2.0.3.ebuild?r1=1.1&r2=1.2

This commit break dependency on FreeBSD. Please depend only when kernel_linux.
Comment 102 William Hubbs gentoo-dev 2011-07-02 17:43:47 UTC
I have adjusted the dependency on makedev so that is only in affect when
stages are being built and for kernel_linux users.
Comment 103 William Hubbs gentoo-dev 2011-07-02 20:59:08 UTC
It turned out that baselayout doesn't need a dependency on makedev at all now since makedev has been added to default/linux/packages.build.
Comment 104 SpanKY gentoo-dev 2011-07-05 06:19:21 UTC
(In reply to comment #91)

i guess you missed the point of "auto".  this bug (as noted by others) is for actually fixing the issue.  if you want to complain, go somewhere else.  if you want to help users, go somewhere else.  if you want to actually fix the problem, then this is the place (but you havent shown anything useful along those lines).

(In reply to comment #103)

i guess we're all set now then ?
Comment 105 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-07-05 19:08:56 UTC
Stages built last week / building this week will finally have the devices back.
The makedev dep was also dropped from baselayout and moved to profiles/default/linux/packages.build.
Comment 106 Gert Andersen 2011-07-06 13:20:45 UTC
(In reply to comment #105)
> Stages built last week / building this week will finally have the devices back.
> The makedev dep was also dropped from baselayout and moved to
> profiles/default/linux/packages.build.

The newest stage3 still not working as i just have notices.
I got over started from clean install of why some og the thing i had setup not like to work for me, so after some days with the first fixes in work after I maually the command and boot up from livecd minimal got used of mount -t proc none /mnt/gentoo
(use of the mount dev command from the handbook of mount --rbind /dev /mnt/gentoo/dev/
chroot /mnt/gentoo /bin/bash 
etc.
USE="build" -1 makedev
and on with it all, this worked fine.
So for a new clean install
mke2fs /dev/sda1
mke2fs -j /dev/sda3
mke2fs -j /dev/sda4
swapon /dec/sda2
mount /dev/sda3 /mnt/gentoo
mkdir /mnt/gentoo/boot
mkdir /mnt/gentoo/home
mount /dev/sds1 /mnt/gentoo/boot
mount /dev/sda4 /mnt/gentoo/home
cd /mnt/gentoo
date 
links http://www.gentoo.org/main/en/mirrors.xlm
gone to inode morror and download here the latest visible stage3-*.tar.bz2 for amd64 (the * just for the date after the - sign)
downloaded then the portage-latest.tar.bz2 file
and untar them as they shall be it.
Then got to 
mount -t proc none /mnt/gentoo/proc
mount --rbind /dev /mnt/gentoo/dev (as the handbook showing it)
do all the other things.
emerge gentoo-soureces
emerge genkernel
run genkernel all
the image file is been compiled 
then
compiling modules
bu after 5 minuttes
error compiling module ...
and this in 6 or more times
genkernel stops
and I seen no initramfs line from genkernel compile
then ls /boot
and no initramfs is to be find
Comment 107 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-07-06 14:05:50 UTC
(In reply to comment #106)

Gert,

your issue doesn't seem related to this bug.
Please open a new bug about it. It also seems you're trying to get some support about the installation, in which case I suggest you try the #gentoo irc channel of the freenode network or the forums https://forums.gentoo.org/

This bug is about an empty /dev in stages. The linked bug is about missing a /etc/runlevels/sysinit/udev symlink in the stages.
Comment 108 Gert Andersen 2011-07-06 14:57:43 UTC
(In reply to comment #107)
> (In reply to comment #106)
> Gert,
> your issue doesn't seem related to this bug.
> Please open a new bug about it. It also seems you're trying to get some support
> about the installation, in which case I suggest you try the #gentoo irc channel
> of the freenode network or the forums https://forums.gentoo.org/
> This bug is about an empty /dev in stages. The linked bug is about missing a
> /etc/runlevels/sysinit/udev symlink in the stages.

It is related to the bug of the missing modules in the udev, that you have write shoudl be fixed in the stage3.
I was trying it out after I formated the partitions sda1 sda2 sda3 sda4, but the only way it could work was by the use of USE="build" emerge -1 makedev, your wrote that coudl be used to fix it if one not would use a new stage3 and the stage3 form the 20.June i got hand of was the latest there was and it wss ot working, but by I reboot the livecd and used the fix command I got it to work.
It is just that if the latest stage3 is from June the 20. is it not the current one there shoudl be fixed yet. I alway look for the latest current stage3 file.
Comment 109 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-07-06 15:57:08 UTC
Gert,

what is the exact name of the stage you were using?
In any case, stages from June 20th are not fixed. I'm going to drop the old stages from the mirrors.
Comment 110 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-07-06 18:25:50 UTC
*** Bug 374179 has been marked as a duplicate of this bug. ***
Comment 111 Mike Nerone 2011-07-06 18:52:33 UTC
(In reply to comment #104)
> i guess you missed the point of "auto".  this bug (as noted by others) is for
> actually fixing the issue.  if you want to complain, go somewhere else.  if you
> want to help users, go somewhere else.  if you want to actually fix the
> problem, then this is the place (but you havent shown anything useful along
> those lines).

You're one of those people who just can't stop attacking someone, huh? I *did* make a *suggestion* for an actual interim fix to the problem. You not liking it doesn't magically make it *not* "a suggestion".

Please drop it. This comment contains no personal attack on you, so does not require a response.
Comment 112 Gert Andersen 2011-07-06 19:45:28 UTC
(In reply to comment #109)
> Gert,
> what is the exact name of the stage you were using?
> In any case, stages from June 20th are not fixed. I'm going to drop the old
> stages from the mirrors.

Jorge,

it is stage3-amd64-20110630.tar.bz2
This was the newest current stage3 I found here on mirros inode in Austria.

There was no newer stage3 file, but it shoudl have been it perhaps.
Comment 113 SpanKY gentoo-dev 2011-07-08 21:07:01 UTC
(In reply to comment #111)

you provably (and by your own admission) provided no suggestion for an actual fix.  throwing up temporary workarounds is not a fix.

no one is talking about attacks personal or otherwise.  if you cant handle being corrected, then that's an issue of your own creation i'm afraid.
Comment 114 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2011-07-08 23:49:56 UTC
(In reply to comment #112)
> (In reply to comment #109)
> > Gert,
> > what is the exact name of the stage you were using?
> > In any case, stages from June 20th are not fixed. I'm going to drop the old
> > stages from the mirrors.
> 
> Jorge,
> 
> it is stage3-amd64-20110630.tar.bz2
> This was the newest current stage3 I found here on mirros inode in Austria.
> 
> There was no newer stage3 file, but it shoudl have been it perhaps.

Gert,

the following is from stage3-amd64-20110707. The stage has the device files and the udev symlink.


atlantis@mini64 ~/abc $ sudo tar xjvf stage3-amd64-20110707.tar.bz2

atlantis@mini64 ~/abc $ ls dev/
console  hda14  hda23  hda32  port    sda15  sdb1   sdb5   sdc13  sdc9   sdd3    tty    tty17  tty26  tty35  tty44  tty53  tty62
core     hda15  hda24  hda4   ptmx    sda2   sdb10  sdb6   sdc14  sdd    sdd4    tty0   tty18  tty27  tty36  tty45  tty54  tty63
fd       hda16  hda25  hda5   random  sda3   sdb11  sdb7   sdc15  sdd1   sdd5    tty1   tty19  tty28  tty37  tty46  tty55  tty7
full     hda17  hda26  hda6   sda     sda4   sdb12  sdb8   sdc2   sdd10  sdd6    tty10  tty2   tty29  tty38  tty47  tty56  tty8
hda      hda18  hda27  hda7   sda1    sda5   sdb13  sdb9   sdc3   sdd11  sdd7    tty11  tty20  tty3   tty39  tty48  tty57  tty9
hda1     hda19  hda28  hda8   sda10   sda6   sdb14  sdc    sdc4   sdd12  sdd8    tty12  tty21  tty30  tty4   tty49  tty58  urandom
hda10    hda2   hda29  hda9   sda11   sda7   sdb15  sdc1   sdc5   sdd13  sdd9    tty13  tty22  tty31  tty40  tty5   tty59  zero
hda11    hda20  hda3   input  sda12   sda8   sdb2   sdc10  sdc6   sdd14  stderr  tty14  tty23  tty32  tty41  tty50  tty6
hda12    hda21  hda30  mem    sda13   sda9   sdb3   sdc11  sdc7   sdd15  stdin   tty15  tty24  tty33  tty42  tty51  tty60
hda13    hda22  hda31  null   sda14   sdb    sdb4   sdc12  sdc8   sdd2   stdout  tty16  tty25  tty34  tty43  tty52  tty61
atlantis@mini64 ~/abc $ ls -la etc/runlevels/sysinit/
total 8
drwxr-xr-x 2 root root 4096 Jul  7 04:50 .
drwxr-xr-x 6 root root 4096 Jul  7 04:50 ..
lrwxrwxrwx 1 root root   17 Jul  7 04:50 devfs -> /etc/init.d/devfs
lrwxrwxrwx 1 root root   17 Jul  7 04:50 dmesg -> /etc/init.d/dmesg
lrwxrwxrwx 1 root root   16 Jul  7 04:50 udev -> /etc/init.d/udev
atlantis@mini64 ~/abc $
Comment 115 Gert Andersen 2011-07-09 08:06:29 UTC
(In reply to comment #114)
> (In reply to comment #112)
> > (In reply to comment #109)
> > > Gert,
> > > what is the exact name of the stage you were using?
> > > In any case, stages from June 20th are not fixed. I'm going to drop the old
> > > stages from the mirrors.
> > 
> > Jorge,
> > 
> > it is stage3-amd64-20110630.tar.bz2
> > This was the newest current stage3 I found here on mirros inode in Austria.
> > 
> > There was no newer stage3 file, but it shoudl have been it perhaps.
> Gert,
> the following is from stage3-amd64-20110707. The stage has the device files and
> the udev symlink.
> atlantis@mini64 ~/abc $ sudo tar xjvf stage3-amd64-20110707.tar.bz2

Yes now have I checked in on 5 mirrors in Europe and now is the latest stage3 that one abowe the same your write it shoudl be, so there must have been a bug or not updated current pointer to the stage3 file when I downloaded it by livecd minimal install cdrom using links http://www.gentoo.org/main/en/mirrors.xlm and moved to mirror inode ftp link I could have used a more nearly link in Denmark ftp.klid..dk
But i got it worked to workded for me and now just have one more gentoo to install now when I have make the last one finished setup as a router pc with all a router server has to use of setup and configured.

> atlantis@mini64 ~/abc $ ls dev/
> console  hda14  hda23  hda32  port    sda15  sdb1   sdb5   sdc13  sdc9   sdd3  
>  tty    tty17  tty26  tty35  tty44  tty53  tty62
> core     hda15  hda24  hda4   ptmx    sda2   sdb10  sdb6   sdc14  sdd    sdd4  
>  tty0   tty18  tty27  tty36  tty45  tty54  tty63
> fd       hda16  hda25  hda5   random  sda3   sdb11  sdb7   sdc15  sdd1   sdd5  
>  tty1   tty19  tty28  tty37  tty46  tty55  tty7
> full     hda17  hda26  hda6   sda     sda4   sdb12  sdb8   sdc2   sdd10  sdd6  
>  tty10  tty2   tty29  tty38  tty47  tty56  tty8
> hda      hda18  hda27  hda7   sda1    sda5   sdb13  sdb9   sdc3   sdd11  sdd7  
>  tty11  tty20  tty3   tty39  tty48  tty57  tty9
> hda1     hda19  hda28  hda8   sda10   sda6   sdb14  sdc    sdc4   sdd12  sdd8  
>  tty12  tty21  tty30  tty4   tty49  tty58  urandom
> hda10    hda2   hda29  hda9   sda11   sda7   sdb15  sdc1   sdc5   sdd13  sdd9  
>  tty13  tty22  tty31  tty40  tty5   tty59  zero
> hda11    hda20  hda3   input  sda12   sda8   sdb2   sdc10  sdc6   sdd14  stderr
>  tty14  tty23  tty32  tty41  tty50  tty6
> hda12    hda21  hda30  mem    sda13   sda9   sdb3   sdc11  sdc7   sdd15  stdin 
>  tty15  tty24  tty33  tty42  tty51  tty60
> hda13    hda22  hda31  null   sda14   sdb    sdb4   sdc12  sdc8   sdd2   stdout
>  tty16  tty25  tty34  tty43  tty52  tty61
> atlantis@mini64 ~/abc $ ls -la etc/runlevels/sysinit/
> total 8
> drwxr-xr-x 2 root root 4096 Jul  7 04:50 .
> drwxr-xr-x 6 root root 4096 Jul  7 04:50 ..
> lrwxrwxrwx 1 root root   17 Jul  7 04:50 devfs -> /etc/init.d/devfs
> lrwxrwxrwx 1 root root   17 Jul  7 04:50 dmesg -> /etc/init.d/dmesg
> lrwxrwxrwx 1 root root   16 Jul  7 04:50 udev -> /etc/init.d/udev
> atlantis@mini64 ~/abc $
Comment 116 Mike Nerone 2011-07-09 09:31:44 UTC
I admitted no such thing - we simply disagree on whether >fix<ing the file itself is a fix. That disagreement doesn't bother me at all - I have lots of friends I disagree on things with. I'm even happy to state that my opinion on that is no more valid than yours. I only took exception to your rudeness in characterizing my comment as a "long rant" and later, your refusal to differentiate opinion from objective fact (there is no such thing as the latter on this subject). Adults can agree to disagree and be civil about it. I'll do my half if you will.
Comment 117 Mike Nerone 2011-07-09 09:35:24 UTC
Criminy, forgot to hit "Reply" (hey look, an objective mistake). The last was supposed to be in reply to comment #113.
Comment 118 Geoff Madden 2011-07-09 16:57:26 UTC
(In reply to comment #117)
> Criminy, forgot to hit "Reply" (hey look, an objective mistake). The last was
> supposed to be in reply to comment #113.

C'mon fellas this's really not the place for this behaviour.
Lets leave it at the place where the mirrors have been updated,and problem solved.

Geoff