Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 282306 - sys-apps/portage-2.2_rc39: UnicodeEncodeError caused by misencoded string passed from portage.dbapi.vartree.dblink._security_check
Summary: sys-apps/portage-2.2_rc39: UnicodeEncodeError caused by misencoded string pas...
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All All
: High normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS, REGRESSION
Depends on:
Blocks:
 
Reported: 2009-08-22 13:22 UTC by Andrey Grozin
Modified: 2009-08-24 10:33 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
handle UnicodeEncodeError bt falling back to utf-8 (security_check.patch,843 bytes, patch)
2009-08-22 19:14 UTC, Zac Medico
Details | Diff
handle UnicodeEncodeError by falling back to utf-8 (security_check.patch,2.06 KB, patch)
2009-08-23 07:49 UTC, Zac Medico
Details | Diff
handle UnicodeEncodeError by falling back to utf-8 (security_check.patch,5.76 KB, patch)
2009-08-24 01:36 UTC, Zac Medico
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Andrey Grozin gentoo-dev 2009-08-22 13:22:37 UTC
>>> Installing (21 of 27) sci-mathematics/Macaulay2-1.2-r3
Traceback (most recent call last):
  File "/usr/bin/emerge", line 40, in <module>
    retval = emerge_main()
  File "/usr/lib/portage/pym/_emerge/main.py", line 1412, in emerge_main
    myopts, myaction, myfiles, spinner)
  File "/usr/lib/portage/pym/_emerge/actions.py", line 457, in action_build
    retval = mergetask.merge()
  File "/usr/lib/portage/pym/_emerge/Scheduler.py", line 820, in merge
    rval = self._merge()
  File "/usr/lib/portage/pym/_emerge/Scheduler.py", line 1125, in _merge
    self._main_loop()
  File "/usr/lib/portage/pym/_emerge/Scheduler.py", line 1255, in _main_loop
    self._poll_loop()
  File "/usr/lib/portage/pym/_emerge/PollScheduler.py", line 127, in _poll_loop
    handler(f, event)
  File "/usr/lib/portage/pym/_emerge/SpawnProcess.py", line 192, in _output_handler
    self._unregister_if_appropriate(event)
  File "/usr/lib/portage/pym/_emerge/AbstractPollTask.py", line 27, in _unregister_if_appropriate
    self.wait()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 42, in wait
    self._wait_hook()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 115, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib/portage/pym/_emerge/EbuildPhase.py", line 84, in _post_phase_exit
    self.wait()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 42, in wait
    self._wait_hook()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 115, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib/portage/pym/_emerge/TaskSequence.py", line 44, in _task_exit_handler
    self.wait()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 42, in wait
    self._wait_hook()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 115, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib/portage/pym/_emerge/CompositeTask.py", line 106, in _default_final_exit
    return self.wait()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 42, in wait
    self._wait_hook()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 115, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib/portage/pym/_emerge/EbuildBuild.py", line 178, in _build_exit
    self.wait()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 42, in wait
    self._wait_hook()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 115, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib/portage/pym/_emerge/CompositeTask.py", line 106, in _default_final_exit
    return self.wait()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 42, in wait
    self._wait_hook()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 115, in _wait_hook
    self._exit_listener_stack.pop()(self)
  File "/usr/lib/portage/pym/_emerge/Scheduler.py", line 1104, in _build_exit
    self._schedule()
  File "/usr/lib/portage/pym/_emerge/PollScheduler.py", line 43, in _schedule
    return self._schedule_tasks()
  File "/usr/lib/portage/pym/_emerge/Scheduler.py", line 1284, in _schedule_tasks
    if q.schedule():
  File "/usr/lib/portage/pym/_emerge/SequentialTaskQueue.py", line 55, in schedule
    task.start()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 24, in start
    self._start()
  File "/usr/lib/portage/pym/_emerge/PackageMerge.py", line 44, in _start
    self.returncode = self.merge.merge()
  File "/usr/lib/portage/pym/_emerge/MergeListItem.py", line 148, in merge
    retval = self._install_task.install()
  File "/usr/lib/portage/pym/_emerge/EbuildBuild.py", line 269, in install
    rval = merge.execute()
  File "/usr/lib/portage/pym/_emerge/EbuildMerge.py", line 27, in execute
    blockers=self.find_blockers)
  File "/usr/lib/portage/pym/portage/__init__.py", line 7360, in merge
    mydbapi=mydbapi, prev_mtimes=prev_mtimes)
  File "/usr/lib/portage/pym/portage/dbapi/vartree.py", line 4179, in merge
    mydbapi=mydbapi, prev_mtimes=prev_mtimes)
  File "/usr/lib/portage/pym/portage/dbapi/vartree.py", line 4189, in _merge
    cleanup=cleanup, mydbapi=mydbapi, prev_mtimes=prev_mtimes)
  File "/usr/lib/portage/pym/portage/dbapi/vartree.py", line 3319, in treewalk
    retval = self._security_check(others_in_slot)
  File "/usr/lib/portage/pym/portage/dbapi/vartree.py", line 3140, in _security_check
    s = os.lstat(path)
  File "/usr/lib/portage/pym/portage/__init__.py", line 172, in __call__
    for x in args]
  File "/usr/lib/portage/pym/portage/__init__.py", line 138, in _unicode_encode
    s = s.encode(encoding, errors)
  File "/usr/lib/python2.6/encodings/koi8_r.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError: 'charmap' codec can't encode character u'\xf6' in position 48: character maps to <undefined>

Reproducible: Always




laptop ~ # emerge --info
Portage 2.2_rc39 (default/linux/x86/2008.0/desktop, gcc-4.4.1, glibc-2.10.1-r0, 2.6.29-tuxonice i686)
=================================================================
System uname: Linux-2.6.29-tuxonice-i686-Intel-R-_Pentium-R-_M_processor_1400MHz-with-gentoo-2.0.1
Timestamp of tree: Sat, 22 Aug 2009 07:15:01 +0000
app-shells/bash:     3.2_p39
dev-java/java-config: 2.1.8-r1
dev-lang/python:     2.5.4-r3, 2.6.2-r1, 3.1.1
dev-python/pycrypto: 2.0.1-r8
dev-util/cmake:      2.6.4
sys-apps/baselayout: 2.0.1
sys-apps/openrc:     0.4.3-r3
sys-apps/sandbox:    2.0
sys-devel/autoconf:  2.13, 2.63-r1
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2, 1.11
sys-devel/binutils:  2.18-r3
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6a
virtual/os-headers:  2.6.27-r2
ACCEPT_KEYWORDS="x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=pentium-m"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-O2 -march=pentium-m"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests collision-protect distlocks fixpackages parallel-fetch preserve-libs protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch"
FFLAGS="-O2 -march=pentium-m"
GENTOO_MIRRORS="ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/ ftp://ftp.uni-erlangen.de/pub/mirrors/gentoo ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo ftp://ftp.wh2.tu-dresden.de/pub/mirrors/gentoo ftp://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/ ftp://ftp.rz.tu-bs.de/pub/mirror/ftp.gentoo.org/gentoo-distfiles/"
LANG="ru_RU.koi8r"
LDFLAGS="-Wl,-O1"
LINGUAS="ru"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/layman/lisp /usr/local/layman/science /usr/local/layman/sunrise /usr/local/layman/x11 /usr/local/portage"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="X a52 aac acl acpi alsa bash-completion berkdb blas bluetooth boehm-gc branding browserplugin bzip2 cairo cdr cli cracklib crypt css cups curl dbus doc dri dts dvd dvdr dvdread eds emacs emboss encode esd evo fam ffmpeg fftw firefox flac fltk fortran gd gdbm gif gmp gnome gpm gsl gstreamer gtk hal iconv imagemagick isdnlog jpeg kde lapack latex ldap leim libnotify mad mikmod mmx mozilla mp3 mp4 mpeg mudflap mule ncurses nls nptl nptlonly nsplugin offensive ogg opengl openmp pam pcre pdf perl plotutils png ppds pppd python qt3 qt3support qt4 quicktime readline reflection schroedinger sdl session speex spell spl ssl startup-notification svg sysfs tcl tcpd tetex tiff tk truetype umfpack unicode usb vcd vorbis win32codecs wxwindows x264 x86 xine xml xorg xulrunner xv xvid zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="ru" USERLAND="GNU" VIDEO_CARDS="vesa intel"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, MAKEOPTS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

laptop ~ # locale
LANG=ru_RU.koi8r
LC_CTYPE="ru_RU.koi8r"
LC_NUMERIC="ru_RU.koi8r"
LC_TIME="ru_RU.koi8r"
LC_COLLATE="ru_RU.koi8r"
LC_MONETARY="ru_RU.koi8r"
LC_MESSAGES="ru_RU.koi8r"
LC_PAPER="ru_RU.koi8r"
LC_NAME="ru_RU.koi8r"
LC_ADDRESS="ru_RU.koi8r"
LC_TELEPHONE="ru_RU.koi8r"
LC_MEASUREMENT="ru_RU.koi8r"
LC_IDENTIFICATION="ru_RU.koi8r"
LC_ALL=
Comment 1 Mounir Lamouri (volkmar) (RETIRED) gentoo-dev 2009-08-22 13:29:07 UTC

*** This bug has been marked as a duplicate of bug 282269 ***
Comment 2 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2009-08-22 14:41:19 UTC
It's a different bug.

(In reply to comment #0)

ru_RU.UTF-8 might allow to workaround this bug.
Comment 3 Zac Medico gentoo-dev 2009-08-22 19:14:24 UTC
Created attachment 201954 [details, diff]
handle UnicodeEncodeError bt falling back to utf-8

If this patch is saved as /tmp/security_check.patch, then it can be applied as follows:

  patch /usr/lib/portage/pym/portage/dbapi/vartree.py /tmp/security_check.patch
Comment 4 Zac Medico gentoo-dev 2009-08-23 07:49:49 UTC
Created attachment 201993 [details, diff]
handle UnicodeEncodeError by falling back to utf-8

This includes the previous patch and should also handle the case shown in bug #282115, comment #6.

If this patch is saved as /tmp/security_check.patch, then it can be applied as
follows:

  patch /usr/lib/portage/pym/portage/dbapi/vartree.py /tmp/security_check.patch
Comment 5 Zac Medico gentoo-dev 2009-08-24 01:36:04 UTC
Created attachment 202099 [details, diff]
handle UnicodeEncodeError by falling back to utf-8

This includes the previous patch and should also handle the case shown in bug
#281199, comment #26.

If this patch is saved as /tmp/security_check.patch, then it can be applied as
follows:

  patch /usr/lib/portage/pym/portage/dbapi/vartree.py /tmp/security_check.patch
Comment 6 Zac Medico gentoo-dev 2009-08-24 06:29:20 UTC
This is fixed in 2.2_rc40.