During the bootstrap.sh, multiple glibc compilations are done, the reasons of which I don't know, but it's happening. Included is /var/log/emerge.log. My process is to USE="-* build bootstrap" emerge --oneshot linux26-headers prior to bootstrapping to bring in only these headers (otherwise, bootstrap.sh brings in 2.4 headers as well), then I bootstrap with USE="nptl" in make.conf. The problem is that because glibc gets compiled multiple times, and bootstrap.sh enables overwriting of any configuration files, even with userlocales specified as a USE flag, multiple locales get written. Since this is mentioned in the installation guide, I think this is a problem. I have livecd in my use flag since I'm building a system for a LiveCD in a chroot... is this the problem? Reproducible: Always Steps to Reproduce: 1. In /etc/make.conf, USE="nptl userlocales livecd" 2. USE="-* build bootstrap" emerge --oneshot linux26-headers 3. scripts/bootstrap.sh 4. locale -a Actual Results: Many extra locales showed up. Expected Results: Only the en_US locales I select should show up. emerge info output, paused in middle of bootstrapping in chroot: Gentoo Base System version 1.4.16 Portage 2.0.51-r3 (default-linux/x86/2004.3, gcc-3.3.4, glibc-2.3.4.20040808-r1, 2.6.9-gentoo-r6 i686) ================================================================= System uname: 2.6.9-gentoo-r6 i686 Intel(R) Pentium(R) 4 CPU 2.40GHz Autoconf: Automake: Binutils: sys-devel/binutils-2.15.90.0.1.1-r3 Headers: sys-kernel/linux26-headers-2.6.8.1 Libtools: ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CFLAGS="-Os -march=i686 -mcpu=i686 -pipe -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" COMPILER="" CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-Os -march=i686 -mcpu=i686 -pipe -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs autoconfig ccache distlocks nodoc noinfo noman sandbox sfperms" GENTOO_MIRRORS="http://mirror.datapipe.net/gentoo" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="" SYNC="rsync://rsync.us.gentoo.org/gentoo-portage" USE="X apm arts avi berkdb bitmap-fonts crypt cups encode f77 foomaticdb fortran gdbm gif gnome gpm gtk gtk2 imlib jpeg kde libg++ libwww livecd mad mikmod motif mpeg ncurses nptl oggvorbis opengl oss pam pdflib png python qt quicktime readline sdl spell ssl svga tcpd truetype userlocales x86 xml2 xmms xv zlib"
For anyone (everyone) currently experiencing this bug, you can either remerge glibc after bootstrapping with the fixed locales.build and the userlocales use flag, or you can replace the locales.build file after each build of glibc in the bootstrap (I think there are only two, so replace after the first so the second time around it still builds the same locales). It's a good idea to use ccache in this case: USE="-* build bootstrap" emerge --nodeps ccache This will hopefully speed up the second round of glibc for the time being until the bug is fixed. Hope this helps someone.
or you use this: * app-admin/localepurge Latest version available: 0.2-r1 Latest version installed: [ Not Installed ] Size of downloaded files: 3 kB Homepage: http://www.gentoo.org/~bass/ Description: Script to recover diskspace wasted for unneeded locale files and localized man pages License: GPL-2 anyway, it doesnt resolves the bug ;)
Actually, no, I think that program is for a different thing - each package installs man pages, documentation, and files for internationalization, and localepurge gets rid of those for other languages. These are the system locales from glibc. Someone with more knowledge can explain exactly what they are.
so when you emerge linux26-headers, glibc isnt built ? in your description you say: then I bootstrap with USE="nptl" in make.conf that's incorrect ... it sounds like you wanted to put USE="nptl userlocales" into make.conf and then bootstrap i dont know why you have 'livecd' in your USE, users should never set that
Sorry if I was ambiguous (and looking back, I probably was), but not quite. First of all, the livecd use flag was because I was attempting to build a LiveCD as per the howto in the Documentation, Tips, and Tricks section, however, it is irrelevant because I encountered this bug on another, NPTL/2.6 headers only bootstrap without that flag. If I don't emerge linux26-headers and I do scripts/bootstrap.sh -f, it starts pulling in linux-headers and fetches linux-2.4.21.tar.bz2 (or at least, it did when I last tried, and assuming it still says to emerge the 26 headers manually prior to bootstrapping in the handbook, I think this is still the behavior). Therefore, I emerge them with USE="-* build bootstrap" emerge --oneshot linux26-headers, which works fine, and then I bootstrap. My USE flags in the second test case were simply "nptl userlocales". Everything compiles perfectly fine. The issue here is that during the course of the bootstrap, glibc (and perhaps others, though I didn't check because none were as significant) is compiled twice. Therefore, it uses the locales.build file I specified the first time through and only compiles the two en_US locales, however, in installing the first pass it replaces the locales.build because cfgprotect is disabled. Therefore, for the second pass, it uses the new locales.build and builds extra locales specified in the example file. Not only is the second compile a pretty large waste of time, especially for people with slow computers where it takes glibc a good few hours to compile, but this creates problems for users who use locales.build but get extra locales and spend extra time compiling the locales anyways. Did this clarify things? If not, please let me know. :)
dunno what docs you've been reading, but we build livecd's with catalyst now (emerge catalyst) any other way is not supported afaik yes, that does clarify things ... we've fixed bootstrap.sh in the past to not compile gcc/glibc twice ... lets see if we can do it again
I was just tooling around with the livecd stuff, here's the page: http://forums.gentoo.org/viewtopic.php?t=244837 Here's a quote: This mini-HowTo will show you how simple it is to create your own LiveCD by hand. It will also allow you to have the following advantages over the catalyst way: 1. The build source will be kept intact and will not get deleted between iterations of LiveCD creation. This will allow you to sync, update, merge, and customize your environment incrementally the same way you do it with a real system. 2. The CD will boot using GRUB, not ISOLINUX. This gives you the same flexibility you have with a real system, that it, changing kernel parameters, discovering devices, etc. 3. As a nice side effect of keeping your build source between iterations, the build time will be reduced dramatically. Yes, catalyst can use pkg's to save time and I'm sure someone could make it use grub, but the ability to work with a real system in a chroot is appealing, and I've tried catalyst before. Honestly, I gave up and did a quick remaster of knoppix - the only reason I needed a new LiveCD was really because I needed drivers for my wireless device which weren't included on the LiveCDs (either the native (crappy) rt2500 drivers or the latest version of ndiswrapper or driverloader)). Still, the issue with the bootstrap remains, and thanks a lot for looking into it. :) At the very least, if you can't figure this one out, could you like, disable config_protect for that one file? I'm not sure, it sounds like a cheap workaround, but there it is. :) Thanks a lot, -Jeff
Well this is particularly annoying when your locale isn't amongst the defaults; you get system which doesn't have support for your locales. Maybe somebody could write a patch which would prevent overwriting of the '/etc/locales.build' config if the 'bootstrap' flag is set?
Yeah, but a patch is a workaround - glibc gets compiled twice, and thats timeconsuming and pointless. Hope you all can fix it. :)
Created attachment 45543 [details] ebuild for the rt2500-1.4.4.0 driver
Comment on attachment 45543 [details] ebuild for the rt2500-1.4.4.0 driver Sorry, I took the wrong bug...
*** Bug 74219 has been marked as a duplicate of this bug. ***
Created attachment 46088 [details] A list of default locales generated I would like to confirm that the problem still persist (as of 2004-12-16). I just did a fresh stage-1 installation from amd64 livecd 2004.3-r1. After finishing stage-3 I noticed that I don't have desired en_US and pl_PL locales. Instead after running 'locale -a' I get a set of default locales (see attachment). As the temporary solution I put "-userlocales" in the USE flags and re-emerged sys-libs/glibc 2.3.4.20040808-r1. Of course, all the above Should Not Happen and Should Not Be Needed :-)
why not just have bootstrap copy the file out of the way like it does with make.conf/make.conf.build
Created attachment 46090 [details] Unnecessary /etc/locales.build file overwrite after glibc emerge Okay, maybe this will be useful. I re-emerged glibc once again. This time I updated /etc/locales.build beforehand to contain only four locales: en_US/ISO-8859-1 en_US.UTF-8/UTF-8 pl_PL/ISO-8859-2 pl_PL.UTF-8/UTF-8 After "emerge glibc" I got a surprising message (see attachment). It looked like after emerging glibc, its ebuild file instructed emerge to overwrite my carefully crafted /etc/locales.build file. Indeed, in the 'glibc-2.3.4.20040808-r1.ebuild' file there are many references to the 'locales.build' file. There are also a couple of lines near the end: <quote> # This is our new config file for building locales insinto /etc doins ${FILESDIR}/locales.build </quote> I suspect that they may be the source of all the trouble. Could someone check it once again and confirm? Regards, Wiktor Wandchowicz
This is expected behavior - someone needs to create the file so that you have something to modify. That's why it comes in the stage tarball. Etc-update is just for this reason: This way, when a package comes with a generic configuration file, it doesn't replace what you have automatically. SpanKY: I like your idea - the problem is, glibc still gets compiled twice, and that's a whole lot of compiling - it takes well over 30 minutes on my p4 2.4 system, and I'm sure there are many with something slower. :)
*** Bug 85675 has been marked as a duplicate of this bug. ***
*** Bug 92729 has been marked as a duplicate of this bug. ***
fixed in portage