When Python is installed, the compiler optimization flags are left out of /usr/lib64/python2.5/config/Makefile (OPT=-NDEBUG but nothing else). When Python packages are installed with emerge, the options are put back in somehow, and things work as expected. But if somebody is installing packages with no ebuild, either as root with python setup.py install or as themselved with python setup.py install --home=$HOME then the packages are compiled completely without any optimization, leading to massive loss of performance. Note that this of course only affects packages with compiled code. I think it was done in this way so that packages installed with emerge are always compiled with the flag in /etc/make.conf, but this side effect is unfortunate. Perhaps the correct solution would be to let Python store all the optimization flags in the OPT variable, and the flags to be removed again before applying the CFLAGS from /etc/make.conf in another variable. Reproducible: Always # emerge --info Portage 2.1.4.5 (default/linux/amd64/2008.0, gcc-4.1.2, glibc-2.6.1-r0, 2.6.25-gentoo-r7 x86_64) ================================================================= System uname: 2.6.25-gentoo-r7 x86_64 Intel(R) Core(TM)2 CPU T5600 @ 1.83GHz Timestamp of tree: Tue, 14 Oct 2008 07:16:01 +0000 app-shells/bash: 3.2_p33 dev-java/java-config: 1.3.7, 2.1.6 dev-lang/python: 2.4.4-r14, 2.5.2-r7 dev-python/pycrypto: 2.0.1-r6 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.2.18.1-r2 sys-devel/autoconf: 2.13, 2.61-r2 sys-devel/automake: 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1-r1 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.5.26 virtual/os-headers: 2.6.23-r3 ACCEPT_KEYWORDS="amd64" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=nocona" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d" CXXFLAGS="-O2 -pipe -march=nocona" DISTDIR="/usr/portage/distfiles" FEATURES="distlocks metadata-transfer parallel-fetch sandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS="http://mirror.uni-c.dk/pub/gentoo/ http://pandemonium.tiscali.de/pub/gentoo/ http://ftp.linux.ee/pub/gentoo/distfiles/ http://trumpetti.atm.tut.fi/gentoo/" LANG="en_US.UTF8" LDFLAGS="-Wl,-O1" MAKEOPTS="-j3" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="X acpi alsa amd64 arts asf atlas bash-completion berkdb bluetooth bzip2 cddb cdparanoia cdr cli cracklib crypt cups dri dvd dvdr dvdread emacs encode firefox foomaticdb fortran gcj gd gdbm gif gimp gpm gs iconv idn imagemagick imap isdnlog java jbig jpeg jpeg2k kdeenablefinal kdexdeltas maildir mbox midi mime mjpeg mmap mmx mp3 mpeg mpi mudflap multilib ncurses netcdf nls nptl nptlonly ogg opengl openmp pam pcmcia pcre pdf perl png ppds pppd python qt3 quicktime readline reflection scanner session smp spell spl sse sse2 ssl sysfs tcpd tetex tiff tk unicode usb vcd wxwindows xine xorg xv xvid xvmc zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="radeon vesa fbdev" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY
No, for once a scripting language is doing The Right Thing (tm), both Ruby and Perl get it wrong.... http://blog.flameeyes.eu/2006/05/19/why-it-is-a-bad-idea-to-record-user-settings-during-compile
I absolutely agree that the behavior *within portage* is correct: If I change CFLAGS in /etc/make.conf and merge a Python package, it is compiled with the correct flags. I do *NOT* suggest changing that, and if fixing this bug would change that behavior then I prefer this bug! But the problem is that distutils rely on people being able to run "python setup.py install" with zero knowledge about compilers and compiler flags on the current architecture and get a package compiled with reasonable compiler flags, and that is not the case now UNLESS run within an ebuild.
(In reply to comment #2) > I absolutely agree that the behavior *within portage* is correct: If I change > CFLAGS in /etc/make.conf and merge a Python package, it is compiled with the > correct flags. I of course meant to write "new flags, which is correct" instead of "correct flags".
I sincerely don't see any difference between distutils and autoconf here, in both cases the user is the one who has to override CFLAGS. Sincerely if a _Gentoo_ user is installing something outside Portage I'd expect it to know how to deal with CFLAGS..
(In reply to comment #4) > I sincerely don't see any difference between distutils and autoconf here, in > both cases the user is the one who has to override CFLAGS. > > Sincerely if a _Gentoo_ user is installing something outside Portage I'd expect > it to know how to deal with CFLAGS.. > I have no experience with autoconf, but usually the idea with distutils is that one can just use it and it does the right thing. Of course most gentoo users can figure out to write env CFLAGS='-O2 -march=nocona' python setup.py build especially since the people doing it most are probably those developing the python package. It is just annoying to have to do it - and in particular to have to tell those using the package to do the same thing if they want to install the package under their own account. IMHO, the correct behaviour would be 1. If CFLAGS environment variable is set, use it. 2. If running under portage, use CFLAGS from /etc/make.conf 3. If none of the above, use the CFLAGS from when Python was compiled. The current behaviour is 1. If CFLAGS environment variable is set, use it. 2. If running under portage, use CFLAGS from /etc/make.conf 3. If none of the above, use CFLAGS="-DNDEBUG" ... which means a sloooow code, with the asserts removed. I have been thinking about what it takes to fix this. Probably some kind of patch to distutils is necessary, which runs the risk of introducing other bugs, so unless somebody really knows what they are doing it is presumable best to leave it as it is. Best regards Jakob
Current behavior is correct.
Re-opening with another proposed behaviour: - leave -DNDEBUG as upstream does, add user flags - add a debug use flag to take -DNDEBUG out.
(In reply to comment #7) > Re-opening with another proposed behaviour: > > - leave -DNDEBUG as upstream does, add user flags > - add a debug use flag to take -DNDEBUG out. While I'm personally inclined to state that upstream screwed up here (to the point where pkgcore/snakeoil code actively mangles distutils flags as needed)... they choose it. Not our place to be forcing behaviour noncompliant with the norm. Take the fight upstream please, reopen here if you get traction there.
