I was just building a new server today using the latest 2004.2 hardened stage1 tarball. Everything was all working until it tried to merge glibc. This is what I encountered: * Checking kernel headers for broken sysctl.h ... yes * Your version of: * /linux/sysctl.h * is broken (from a user space perspective). Please apply * the following patch: * ******************************************************* cat: /var/tmp/portage-pkg/glibc-2.3.3.20040420-r1/inf/files/fix-sysctl_h.patch: No such file or directory * ******************************************************* * To fix, just do this: * cd /linux/ * patch -p3 < /var/tmp/portage-pkg/glibc-2.3.3.20040420-r1/inf/files/fix-sysctl_h.patch !!! ERROR: glibc-2.3.3.20040420-r1/glibc-2.3.3.20040420-r1 failed. !!! Function pkg_setup, Line 214, Exitcode 0 !!! Broken linux/sysctl.h header included in kernel sources! !!! Error running pkg_setup The effected code in the ebuild seems to be this area and I'd like to know who put that there and if they could please fix it. :) if [ "$(KV_to_int $(uname -r))" -gt "`KV_to_int '2.5.68'`" ] then local KERNEL_HEADERS="$(get_KHV "`KV_to_int ${MIN_NPTL_KV}`")" einfon "Checking kernel headers for broken sysctl.h ... " if ! gcc -I"${KERNEL_HEADERS}" \ -c ${FILESDIR}/test-sysctl_h.c -o ${T}/test1.o &> /dev/null then echo "yes" Right now I'm stuck at not finishing a stage1 build and don't know where to go. That einfo says to patch that file, but if I do that and rerun bootstrap.sh won't that be overwritten again? Also, Why can't it fix that file in the first place? This all just seems quite odd to me. Let me know what I can do to help resolve this. Thanks! livecd portage # emerge info Portage 2.0.50-r10 (hardened-x86-2004.0, gcc-3.3.3, glibc-2.3.3.20040420-r0, 2.6.7-gentoo-r11) ================================================================= System uname: 2.6.7-gentoo-r11 i686 Pentium III (Katmai) Gentoo Base System version 1.4.16 Autoconf: Automake: ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CFLAGS="-march=pentium3 -O2 -pipe" CHOST="i686-pc-linux-gnu" COMPILER="" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-march=pentium3 -O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs buildpkg ccache sandbox sfperms strict" GENTOO_MIRRORS="http://gentoo.osuosl.org/ http://gentoo.mirrors.pair.com/" MAKEOPTS="-j4" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="berkdb crypt hardened mmx ncurses pam perl pic pie python readline snmp ssl tcpd x86 xml"
I redid the bootstrap again and the same thing happened. This time I'm posting more of the backlog to show where this fails.. Which is during the merge, while the error is in pkg_setup. Quite odd. ./lib/libpcprofile_pic.a ./lib/libSegFault_pic.a ./lib/libthread_db_pic.a ./lib/libnss_hesiod_pic.a ./lib/libnsl_pic.a ./lib/libnss_nis_pic.a ./lib/libnss_nisplus_pic.a ./lib/libnss_compat_pic.a ./lib/libutil_pic.a ./lib/libc_pic.a ./lib/libanl_pic.map ./lib/libc_pic.map ./lib/libdl_pic.map ./lib/libm_pic.map ./lib/libnsl_pic.map ./lib/libnss_dns_pic.map ./lib/libnss_files_pic.map ./lib/libnss_hesiod_pic.map ./lib/libnss_nis_pic.map ./lib/libnss_nisplus_pic.map ./lib/libpthread_pic.map ./lib/libresolv_pic.map ./lib/librt_pic.map ./lib/librtld_pic.map ./lib/libutil_pic.map ./lib/libthread_db_pic.map ./lib/ld_pic.map ./lib/libBrokenLocale_pic.map ./lib/libcrypt_pic.map ./lib/libnss_compat_pic.map ./lib/librtld.os_pic.map ./etc/ ./etc/rpc ./etc/nscd.conf ./sbin/ ./sbin/ldconfig ./sbin/sln >>> Done. >>> extracting info * Checking kernel headers for broken sysctl.h ... yes * Your version of: * /linux/sysctl.h * is broken (from a user space perspective). Please apply * the following patch: * ******************************************************* cat: /var/tmp/portage-pkg/glibc-2.3.3.20040420-r1/inf/files/fix-sysctl_h.patch: No such file or directory * ******************************************************* * To fix, just do this: * cd /linux/ * patch -p3 < /var/tmp/portage-pkg/glibc-2.3.3.20040420-r1/inf/files/fix-sysctl_h.patch !!! ERROR: glibc-2.3.3.20040420-r1/glibc-2.3.3.20040420-r1 failed. !!! Function pkg_setup, Line 214, Exitcode 0 !!! Broken linux/sysctl.h header included in kernel sources! !!! Error running pkg_setup real 152m1.455s user 205m51.244s sys 38m26.839s livecd portage #
Created attachment 38300 [details] emerge debug output Here's some output from running emerge -d glibc. Please take a look at see what happens. I'll be afk most of the day today, so I won't be able to test anything new. Talking to solar last night, he wondered if it was related to a portage bug concerning pkg_setup. Somehow pkg_setup is getting run after a merge. He and I couldn't remember the bug number, but it might be worth finding/asking about. Tseng: were you running the same version of portage as me when you tested that build?
I've done a build from the same stage a dozen times or more, just verified yesterday that everything bootstraps cleanly. According to Ramereth, whom I dont doubt, his error message (clearly in pkg_setup), comes after src_compile. Portage buglet? The Man says: <@solar> that could also stem from http://bugs.gentoo.org/show_bug.cgi?id=59667 <@solar> but I'm pretty sure there is a bug with portage not exporting envvars... ferringb knowsmore about it.
Ok, I just tested the same thing on a P4 that already has Gentoo up and running (all I did was test inside a chroot) and everything built fine which is odd considering this has failed for two days in a row on a livecd on this other box. Is there anything on how the livecd is setup that could cause this error rather than doing it inside a chroot on an already running gentoo machine? I'm at a loss for that answer.
*** Bug 61890 has been marked as a duplicate of this bug. ***
Created attachment 38491 [details] chroot_bootstrap_glibc.log.gz Same problem here from stage 1 (non hardened). Seems to be a post-packaging problem in pkg_setup, the log is an strace excerpt of chrooted emerge -K glibc. It was run after glibc got packaged during bootstrap and failed to merge it to /. I trapped down the problem to be in the FILESDIR variable, which is: FILESDIR=/tmp/portage/portage-pkg/glibc-2.3.3.20040420-r1/inf/files so gcc cannot find the test sources. I included them in the ebuild, ebuild patch to -r1 will follow once glibc emerged successfully on the boostrap-failed system.
Created attachment 38492 [details, diff] glibc-2.3.3.20040420-r1.ebuild.diff It worked, so this is the patch against glibc-2.3.3.20040420-r1.ebuild
Created attachment 38493 [details] glibc-2.3.3.20040420-r1.ebuild In case you are not familar with patch, this is the full .ebuild.
That fix seems to be more of a hack than a real fix. That doesn't address the reason why ${FILESDIR} is incorrect in the first place. Plus, from what I understand, pkg_setup shouldn't be run after the merge. That should be ran before the package is compiled. Please correct me if I'm wrong! I'm going to try this ebuild for now to see if I can at least get this box built, but like I said, there is still something funky going on with environment variables in that ebuild.
Yes, so far it's only a hack. I guess the bug is in portage itself? %( According to man 5 ebuild, FILESDIR="${PORTDIR}/${CATEGORY}/${PN}/files". So that used in pkg_setup is definitly wrong, because ${PORTDIR}=/usr/portage (see log.gz before). In the log, it looks like FILESDIR="${O}/files", but i found no word about O in the documentation. The manpage also says, that pkg_setup is for "actions or checks to be preformed before anything else.", i.e. it should also be executed if you emerge a .tbz (packaged ebuild). So the behaviour should be right, though the check before merging is not necessary if merging is done straight after compiling. PS: Correct me if i got something wrong, i'm not a gentoo developer... :-)
Created attachment 38518 [details] emerge_info.txt I guess the following, because my FEATURES also contains buildpkg (see emerge_info.txt): When emerging a packaged ebuild, the .ebuild file temporarily goes somewhere into PORTAGE_TMPDIR before it finally gets merged into /var/db/pkg and the dirname of that temporary .ebuild (probably stored in envvar O) is used for constructing the FILESDIR variable, resulting in a wrong FILESDIR. Could anybody with a faster machine or having python knowledge verify, that portage behaves like this? I.e. test, if the first succeeds and the second fails: 1. FEATURES="-buildpkg" emerge glibc 2. FEATURES="buildpkg" emerge glibc or have a look at the portage sources for setting the FILESDIR variable? If this is a case, please also file a portage bug... :-)
Created attachment 38519 [details] emerge info output
I have a similar problem on a running system. I installed a new machine yesterday, using the 2004.2 stage1 for x86. All went fine. After base installation completed I did an emerge sync to install xorg. Then I tried an emerge -puv system which wants to install glibc-2.3.3-20040420-r1. The package compiled succesfully, and at the beginning it states, that my syscntl.h is not(!) broken. But at the merge state, it does another check of syscntl.h and then fails with the error described herein. Maybe this will help to find the error. for emerge info see attachment emerge info output.
EBUILD BUG -- My thoughts: FILESDIR is not a valid place to look with BINARY packages. Those experiencing the problem more than likely have FEATURES=buildpkg set and the ebuild is trying to use the repository for the patch and getting a different FILESDIR than one would have set when compiling. FEATURES=buildpkg Results in: unpack, compile, package, -->merge package<-- At which point there should be NO accessing of the portage tree. Final verdict: Do not use O, FILESDIR or any other related variable when past the install stage .of an ebuild.
As Ramereth has pointed out the pkg_setup part... I'm still working on an answer that is more reasonable. Do not depend on filesdir in the setup function. It gets called for every invocation of ebuild.sh and thus in the merge phase it breaks.
I just confirmed that: FEATURES="-buildpkg" emerge glibc Does infact bypass this bug. Nick: thanks for pointing out those errors and insight into the problem! Hopefully we can narrow down what exactly is causing this.
After a long night of bootstrapping (it's a 800mhz via C3 :) I can confirm that the first-mentioned patch, does indeed fix the problem :) I'll continue, and see what else turns up, on my new hardened box :) Thanks for your insight.
*** Bug 62269 has been marked as a duplicate of this bug. ***
This is really due to the fact that the ebuild script for kernels -gt 2.5.58 tries to check that the /usr/src/linux/include/linux/sysctl.h file can be used from user space but at least with 2.6.7 and later kernel headers it can never be because /usr/src/linux/include/linux/list.h completely forbids this with the condition #ifdef __KERNEL_ ... #else #warning "don't include kernel headers in userspace" #endif /* __KERNEL__ */ so either the ebuild is wrong now with 2.6.x kernel header files or the kernel header needs to be fixed. The 2.4.x list.h kernel header file does not have this warning conditional.
the problem comes in that the linux-headers installed in /usr/include are too old so the ebuild moves to trying to use the /usr/src/linux/include headers ... so if your linux-headers are setup properly, the ebuild shouldnt try to use /usr/src/linux
Created attachment 38714 [details, diff] glibc-pkg_setup.diff This should solve the problem here. We rename the pkg_setup() function to glibc_setup() and only call glibc_setup() when we are in the src_unpack() We also disable the FORCE_DOWNGRADE check as well. It's fails on the vercmp.
genone: Can you please fix the python FORCE_DOWNGRADE check? We had to temp disable in a few revision of glibc (that's your code right?) lance: Can you patch your existing ebuild and try again? If the bodx has already been worked around and/or put into production then could you please test in a chroot with the glibc-pkg_setup.diff posted here. cd $(portageq envvar PORTDIR)/sys-libs/glibc cvs update -d -P wget -q -O - 'http://bugs.gentoo.org/attachment.cgi?id=38714&action=view' | patch ebuild glibc-2.3.3.20040420-r1.ebuild digest ebuild glibc-2.3.4.20040619-r1.ebuild digest ACCCEPT_KEYWORDS=~x86 emerge glibc -pv # Motivation stems from comment #14
sigh... Ok I updated it anyway.
