Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 280001 - dev-lang/python-3.1 fails to build with sys-libs/db headers containing non-UTF-8 characters
Summary: dev-lang/python-3.1 fails to build with sys-libs/db headers containing non-UT...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Unspecified (show other bugs)
Hardware: All All
: High normal (vote)
Assignee: Python Gentoo Team
URL:
Whiteboard:
Keywords:
: 280600 (view as bug list)
Depends on:
Blocks:
 
Reported: 2009-08-02 08:09 UTC by Priit Laes (IRC: plaes)
Modified: 2009-08-10 14:47 UTC (History)
9 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Priit Laes (IRC: plaes) 2009-08-02 08:09:48 UTC
running build
running build_ext
Traceback (most recent call last):
  File "./setup.py", line 1688, in <module>
    main()
  File "./setup.py", line 1683, in main
    "Tools/scripts/2to3"]
  File "/var/tmp/portage/dev-lang/python-3.1/work/Python-3.1/Lib/distutils/core.py", line 149, in setup
    dist.run_commands()
  File "/var/tmp/portage/dev-lang/python-3.1/work/Python-3.1/Lib/distutils/dist.py", line 919, in run_commands
    self.run_command(cmd)
  File "/var/tmp/portage/dev-lang/python-3.1/work/Python-3.1/Lib/distutils/dist.py", line 938, in run_command
    cmd_obj.run()
  File "/var/tmp/portage/dev-lang/python-3.1/work/Python-3.1/Lib/distutils/command/build.py", line 128, in run
    self.run_command(cmd_name)
  File "/var/tmp/portage/dev-lang/python-3.1/work/Python-3.1/Lib/distutils/cmd.py", line 315, in run_command
    self.distribution.run_command(command)
  File "/var/tmp/portage/dev-lang/python-3.1/work/Python-3.1/Lib/distutils/dist.py", line 938, in run_command
    cmd_obj.run()
  File "/var/tmp/portage/dev-lang/python-3.1/work/Python-3.1/Lib/distutils/command/build_ext.py", line 347, in run
    self.build_extensions()
  File "./setup.py", line 112, in build_extensions
    missing = self.detect_modules()
  File "./setup.py", line 742, in detect_modules
    f = open(f).read()
  File "/var/tmp/portage/dev-lang/python-3.1/work/Python-3.1/Lib/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 917: ordinal not in range(128)
make: *** [sharedmods] Error 1
 * 
 * ERROR: dev-lang/python-3.1 failed.
 * Call stack:
 *               ebuild.sh, line   49:  Called src_compile
 *             environment, line 3937:  Called _eapi2_src_compile
 *               ebuild.sh, line  634:  Called die
 * The specific snippet of code:
 *   		emake || die "emake failed"
 *  The die message:
 *   emake failed
 * 
 * If you need support, post the topmost build error, and the call stack if relevant.
 * A complete build log is located at '/var/tmp/portage/dev-lang/python-3.1/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/dev-lang/python-3.1/temp/environment'.
 * 

chi ~ # emerge --info
Portage 2.2_rc33 (default/linux/amd64/2008.0/no-multilib, gcc-4.3.3, glibc-2.10.1-r0, 2.6.31-rc4 x86_64)
=================================================================
System uname: Linux-2.6.31-rc4-x86_64-Intel-R-_Core-TM-2_CPU_L7400_@_1.50GHz-with-gentoo-2.0.1
Timestamp of tree: Sun, 02 Aug 2009 07:00:02 +0000
ccache version 2.4 [enabled]
app-shells/bash:     4.0_p28
dev-java/java-config: 1.3.7-r1, 2.1.8-r1
dev-lang/python:     2.6.2-r1
dev-python/pycrypto: 2.0.1-r8
dev-util/ccache:     2.4-r8
dev-util/cmake:      2.6.4-r2
sys-apps/baselayout: 2.0.1
sys-apps/openrc:     0.4.3-r3
sys-apps/sandbox:    2.0
sys-devel/autoconf:  2.13, 2.63-r1
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2, 1.11
sys-devel/binutils:  2.19.1-r1
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6a
virtual/os-headers:  2.6.30-r1
ACCEPT_KEYWORDS="amd64 ~amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -ggdb -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/eselect/postgresql /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-march=native -ggdb -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="ccache collision-protect distlocks fixpackages multilib-strict parallel-fetch preserve-libs protect-owned sandbox sfperms splitdebug strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LDFLAGS="-Wl,--as-needed"
LINGUAS="en et"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/home/plaes/code/gentoo/gnome /home/plaes/code/gentoo/local /home/plaes/code/gentoo/tryton-overlay"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X a52 aac acpi adns alsa amd64 amr apache2 ares autoipd avahi berkdb blas bluetooth branding bzip2 cairo cdr cjk cli consolekit cracklib crypt cups curl daap dbus dga djvu dri dvd dvdr dvi eds encode exif faac faad fam ffmpeg fftw flac fortran fuse galago gcj gd gdbm gif gimp git glib glitz gmp gnome gnome-keyring gnuplot gnutls gphoto2 gs gsl gstreamer gtk hal hardened hdri iconv idn imagemagick ipv6 isdnlog jingle jpeg jpeg2k kdrive kpathsea lapack laptop latex lcms libffi libnotify lzma lzo mad mmx mng moonlight mozdevelop mp3 mpeg msn mudflap music nautilus ncurses network network-cron networkmanager nls nptl nptlonly nsplugin numpy ogg openexr opengl openmp pam pcmcia pcre pcsc-lite pdf perl png policykit pppd pygments python qt3support raw readline reflection resolvconf samba sasl sdl session smartcard smp sound source spl sse sse2 ssl ssse3 subversion svg sysfs tcpd theora tiff truetype unicode usb v4l v4l2 vim-syntax vorbis webkit wps x264 xcb xinerama xml xmp xorg xulrunner xv xvid xvmc zeroconf zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" CAMERAS="adc65 agfa_cl20 aox barbie canon casio_qv clicksmart310 digigr8 digita dimagev dimera3500 directory enigma13 fuji gsmart300 hp215 iclick jamcam jd11 jl2005a kodak_dc120 kodak_dc210 kodak_dc240 kodak_dc3200 kodak_ez200 konica konica_qm150 largan lg_gsm mars mustek panasonic_coolshot panasonic_dc1000 panasonic_dc1580 panasonic_l859 pccam300 pccam600 polaroid_pdc320 polaroid_pdc640 polaroid_pdc700 ptp2 ricoh ricoh_g3 samsung sierra sipix_blink sipix_blink2 sipix_web2 smal sonix sony_dscf1 sony_dscf55 soundvision spca50x sq905 stv0674 stv0680 sx330z template topfield toshiba_pdrm11" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en et" USERLAND="GNU" VIDEO_CARDS="intel"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Alex HeadHunter Pyattaev 2009-08-02 09:18:42 UTC
Hello! 
I can confirm the bug with exactly the same fail location on AMD64:
Portage 2.1.6.13 (default/linux/amd64/2008.0/desktop, gcc-4.3.3, glibc-2.10.1-r0, 2.6.30-gentoo-r4-localbuild x86_64)
Comment 2 Nikolay D Hodyunya 2009-08-02 12:59:53 UTC
same bug

Comment 3 Yuriy Dmitriev 2009-08-02 17:43:47 UTC
I also confirm this bug.
Comment 4 James Earl Spahlinger 2009-08-02 19:16:02 UTC
Please be sure that you have run 'python-updater', assigning to python herd due to multiple confirmations, thanks for reporting.
Comment 5 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2009-08-02 19:21:18 UTC
Post the output of:
emerge -ptv python:3.1
Comment 6 Rottex 2009-08-03 00:02:25 UTC
Same here. AMD64

& if I run python-updater it finds some packages but immed. wants to emerge 3.1 as first action

so I'm stuck here....

----------------------------------------------------

emerge -ptv python:3.1

These are the packages that would be merged, in reverse order:

Calculating dependencies                      ... done!
[ebuild  NS   ] dev-lang/python-3.1 [2.5.4-r2, 2.6.2-r1] USE="gdbm ncurses readline ssl threads xml -build -doc -examples -ipv6 -sqlite -tk -ucs2 -wininst" 0 kB

Total: 1 package (1 in new slot), Size of downloads: 0 kB
Comment 7 Rottex 2009-08-03 01:39:14 UTC
FYI:
compiling by hand with debug on found out that in 
/usr/include/db4.5/db.h char 917 
and
/usr/include/db4.6/db.h char 902 
was März (german word for March) in the 
#define DB_VERSION_STRING   "Berkeley DB 4.6.21: (März 24, 2009)"

equery belongs /usr/include/db4.6/db.h
 * Searching for /usr/include/db4.6/db.h ...
sys-libs/db-4.6.21_p4 (/usr/include/db4.6/db.h)

so it seems that sys-libs/db renders the version string in german which causes no issues when in Juli , August and so on...

I changed LC_ALL to en_us 

LC_ADDRESS=de_DE.UTF-8
LC_ALL=en_US.UTF-8
LC_COLLATE=de_DE.UTF-8
LC_CTYPE=en_US.UTF-8
LC_IDENTIFICATION=de_DE.UTF-8
LC_MEASUREMENT=de_DE.UTF-8
LC_MESSAGES=en_US.UTF-8
LC_MONETARY=de_DE.UTF-8
LC_NAME=de_DE.UTF-8
LC_NUMERIC=de_DE.UTF-8
LC_PAPER=de_DE.UTF-8
LC_TELEPHONE=de_DE.UTF-8
LC_TIME=de_DE.UTF-8

and reemerged sys-libs/db
but realized that we have August :-) (english and german the same...) right now which doesn't trigger a umlaut and I can't change the time on the server easily... 

temporary fix replace "ä" with "a" in the db.h files
Comment 8 Kraml Liu 2009-08-03 02:18:01 UTC
Yes I can confirm this. I'm Chinese so the version string in db.h is always different than in English and is not in the ASCII range, so python 3.1 will fail to emerge.

My workaround is to re-emerge db with 
LANG=C emerge -1 sys-libs/db

or put "LANG=C" in the file /etc/portage/env/sys-libs/db and then re-emerge db.

After re-emerge sys-libs/db, python 3.1 emerged with no error.

(In reply to comment #7)
> FYI:
> compiling by hand with debug on found out that in 
> /usr/include/db4.5/db.h char 917 
> and
> /usr/include/db4.6/db.h char 902 
> was März (german word for March) in the 
> #define DB_VERSION_STRING   "Berkeley DB 4.6.21: (März 24, 2009)"
> 
> equery belongs /usr/include/db4.6/db.h
>  * Searching for /usr/include/db4.6/db.h ...
> sys-libs/db-4.6.21_p4 (/usr/include/db4.6/db.h)
> 
> so it seems that sys-libs/db renders the version string in german which causes
> no issues when in Juli , August and so on...
> 
> I changed LC_ALL to en_us 
> 
> LC_ADDRESS=de_DE.UTF-8
> LC_ALL=en_US.UTF-8
> LC_COLLATE=de_DE.UTF-8
> LC_CTYPE=en_US.UTF-8
> LC_IDENTIFICATION=de_DE.UTF-8
> LC_MEASUREMENT=de_DE.UTF-8
> LC_MESSAGES=en_US.UTF-8
> LC_MONETARY=de_DE.UTF-8
> LC_NAME=de_DE.UTF-8
> LC_NUMERIC=de_DE.UTF-8
> LC_PAPER=de_DE.UTF-8
> LC_TELEPHONE=de_DE.UTF-8
> LC_TIME=de_DE.UTF-8
> 
> and reemerged sys-libs/db
> but realized that we have August :-) (english and german the same...) right now
> which doesn't trigger a umlaut and I can't change the time on the server
> easily... 
> 
> temporary fix replace "ä" with "a" in the db.h files
> 

Comment 9 Alex HeadHunter Pyattaev 2009-08-03 10:30:49 UTC
OK, I have tried the proposed fix and it appears to work fine. However, python-updater thinks that I should update to 2.6: "* Starting Python Updater from 3.1 to 2.6". Nice, really.
Comment 10 Stoian Ivanov 2009-08-03 18:25:30 UTC
if you have sloted sys-libs/db you need to rebuild all of them with LANG=C 
hint: equery list sys-libs/db
Comment 11 Alex Pogodin 2009-08-03 21:24:50 UTC
Same problem here, LANG=ru_RU.UTF-8. Re-emerging sys-libs/db helps building python-3.1,  but python-updater switches to 2.6. I have no slotted ebuilds installed:

# equery list sys-libs/db
 * Searching for db in sys-libs ...
[IP-] [ ~] sys-libs/db-4.7.25_p4 (4.7)
Comment 12 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2009-08-04 05:49:28 UTC
This bug should be now fixed in dev-lang/python-3.1.

Maintainers of sys-libs/db:
You might want to add setting of LC_ALL="C" to build system of sys-libs/db.
Comment 13 Alex HeadHunter Pyattaev 2009-08-04 18:56:28 UTC
It appears that the old emerge will not work with python 3. I have looked through the ChangeLog and found out that there are MASSIVE syntax restrictions added. So, we will not see python3 as default python interpreter any time soon (at least until whole portage is rewritten along with other python-based system proggies). So, I propose you DO NOT run python-updater at all and DO NOT switch to python 3 as default interpreter at least for now. 
Comment 14 Justin Lecher (RETIRED) gentoo-dev 2009-08-06 21:16:02 UTC
*** Bug 280600 has been marked as a duplicate of this bug. ***
Comment 15 Paul de Vrieze (RETIRED) gentoo-dev 2009-08-10 13:37:10 UTC
I'm currently looking at this and figuring out where to fix it. It seems that only the install phase needs to be fixed. In the meantime, I'd like some feedback on which versions of db need to be fixed. I could fix them all and revbump them all, or I could fix them all and just revbump 4.7. Or I could just fix 4.7 and 4.6. In the meantime, while localized version strings are dubious, I think python is a bit braindead as well in actually parsing the stuff as it depends on the actual build date, and thus is variable.
Comment 16 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2009-08-10 14:47:09 UTC
(In reply to comment #15)

I suggest this line in src_unpack():

sed -e "/^DB_RELEASE_DATE=/s/%B %e, %Y/%Y-%m-%d/" -i dist/RELEASE