emerge, little patch, major speedup Hi, I set up a crosscompiler on my athlon-xp and set up distcc on athlon-xp and alpha. My experience is, configure is very very slow... and I found out why!!! UID PID PPID C STIME TTY TIME CMD root 8936 31285 4 14:01 pts/1 00:00:08 /usr/bin/python /usr/bin/emerge -B net-misc/rsync root 9072 8936 1 14:01 pts/1 00:00:02 /bin/bash /usr/sbin/ebuild.sh compile root 9109 9072 3 14:01 pts/1 00:00:04 /bin/sh ./configure --prefix=/usr --host=alpha-unknown-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --without-included-popt root 11138 9109 0 14:04 pts/1 00:00:00 /bin/sh ./configure --prefix=/usr --host=alpha-unknown-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --without-included-popt root 11139 11138 0 14:04 pts/1 00:00:00 alpha-unknown-linux-gnu-gcc -E conftest.c root 11140 11139 0 14:04 pts/1 00:00:00 alpha-unknown-linux-gnu-gcc -E conftest.c root 11141 11140 6 14:04 pts/1 00:00:00 /bin/bash /usr/bin/gcc-config --get-bin-path root 11142 11141 0 14:04 pts/1 00:00:00 /bin/bash /usr/bin/gcc-config --get-bin-path root 11143 11142 66 14:04 pts/1 00:00:02 python -c import portage; print portage.settings["NOCOLOR"] See output of ps -efww, alpha-unknown-linux-gnu-gcc calls gcc-config which calls python to get the NOCOLOR setting. Now python takes ages to get this setting and is done for every configure step using gcc. Now, gcc-config sources /etc/init.d/functions, and here the python command is called: RC_NOCOLOR="$(python -c 'import portage; print portage.settings["NOCOLOR"]' 2> /dev/null)" Now, changing this into #RC_NOCOLOR="$(python -c 'import portage; print portage.settings["NOCOLOR"]' 2> /dev/null)" RC_NOCOLOR="false" gives me a major speedup! I tested this little patch with the command: # time emerge -B net-misc/rsync Results: without distcc, without patch ----------------------------- real 28m51.139s user 25m54.925s sys 2m38.936s without distcc, with this little patch -------------------------------------- real 9m12.673s user 7m25.182s sys 1m37.482s with distcc, without patch -------------------------- real 27m44.437s user 24m42.533s sys 2m45.595s with distcc, with this little patch ----------------------------------- real 7m48.909s user 5m33.383s sys 1m43.455s You see, a speedup of upto 3.55 times on my Alpha XL 266 Oh and by the way, can sys-apps/time-1.7-r1 finally be marked as stable for alpha? Thanx! Gentoo Base System version 1.4.3.10 Portage 2.0.49-r21 (default-alpha-1.4, gcc-3.3.2, glibc-2.3.2-r9, 2.4.24) ================================================================= System uname: 2.4.24 alpha EV4 distcc 2.11.1 alpha-unknown-linux-gnu (protocols 1 and 2) (default port 3632) [enabled] ACCEPT_KEYWORDS="alpha" AUTOCLEAN="yes" CFLAGS="-O2 -pipe " CHOST="alpha-unknown-linux-gnu" COMPILER="gcc3" CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d" CXXFLAGS="-O2 -pipe " DISTDIR="/data/linux/gentoo/distfiles" FEATURES="ccache distcc" GENTOO_MIRRORS="http://www.mirror.ac.uk/sites/www.ibiblio.org/gentoo http://ftp.easynet.nl/mirror/gentoo http://gentoo.oregonstate.edu http://www.ibiblio.org/pub/Linux/distributions/gentoo" MAKEOPTS="-j4" PKGDIR="/data/linux/gentoo/packages/alpha" PORTAGE_TMPDIR="/small/portage_tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync4.nl.gentoo.org/gentoo-portage" USE="X alpha alsa berkdb cdr crypt cups encode esd foomaticdb gdbm gif gnome gpm gtk gtk2 imlib jpeg libg++ libwww mikmod motif mozilla ncurses oggvorbis opengl oss pam pdflib perl png python quicktime readline sdl slang ssl tcpd truetype xml2 xmms xv zlib"
This does not only affect alpha, this affects all archs. AMD Athlon XP 2600+ only doesn't have such a big speed difference since the machine itself is so bloody fast!
make sure your gcc-config is up-to-date *** This bug has been marked as a duplicate of 27618 ***
Have a look at /usr/portage/sys-devel/gcc-config/files/wrapper-1.4.1.c. If gcc-config is called, that means it could not locate gcc through the env.d file _or_ the PATH. I would rather check why that is, as just calling gcc-config also slow things a lot.
I think this bug is not a duplicate of 27618 and it still exists! I'm using gcc-config-1.3.4 both on Alpha and Athlon XP. To trace the problem I added a line to the beginning of /usr/bin/gcc-config containing: date '+%Y.%m.%d %H:%M:%S' >>/tmp/gcc-config.trace Normally gcc-config doesn't get called, but when I do the following: on Alpha: cd /etc/env.d/gcc vi alpha-unknown-linux-gnu-3.3.2 change CC=gcc into CC=alpha-unknown-linux-gnu-gcc change CXX=g++ into CXX=alpha-unknown-linux-gnu-g++ gcc-config alpha-unknown-linux-gnu-3.3.2 env-update . /etc/profile echo $PATH /bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/opt/bin:/usr/alpha-unknown-linux-gnu/gcc-bin/3.3:/usr/X11R6/bin on Athlon XP: cd /etc/env.d/gcc vi i686-pc-linux-gnu-3.2.3 change CC=gcc into CC=i686-pc-linux-gnu-gcc change CXX=g++ into CXX=i686-pc-linux-gnu-g++ gcc-config i686-pc-linux-gnu-3.2.3 env-update . /etc/profile echo $PATH /bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/3.2:/usr/X11R6/bin:/opt/blackdown-jdk-1.4.1/bin:/opt/blackdown-jdk-1.4.1/jre/bin:/usr/qt/3/bin:/usr/kde/3.1/sbin:/usr/kde/3.1/bin:/var/qmail/bin:/home/crossdev/alpha/bin on both: > /tmp/gcc-config.trace emerge -B net-misc/rsync # cat /tmp/gcc-config.trace | wc -l When FEATURES="-cache -distcc", /tmp/gcc-config.trace contains on Alpha 213 lines, on Athlon XP 215 lines, and when I enable ccache, the number of lines is even bigger! This also happens when I put distcc in FEATURES which is the main reason for changing /etc/env.d/gcc/alpha-unknown-linux-gnu-3.3.2. Am I doing something wrong or is this a bug?
This is not alpha specific at all. Azarah, please take a look at this. It looks like Evert has done some real research and may have found a real problem. (I'm not familiar enough with gcc-config to say one way or the other)
I looked. Seems it is only and issue if you call $CHOST-gcc or whatever, as the wrapper did not consider symlinks to be valid. Should be fixed in 1.3.5.
