Created attachment 379146 [details] python code snippet For a specific linux kernel source file (./drivers/staging/rtl8821ae/btcoexist/HalBtc8812a1Ant.c) I do get at a x86 Gentoo: UnicodeDecodeError: 'utf8' codec can't decode byte 0xb3 in position 47021: invalid start byte Happens with python 2.7.6 and 3.3.3 $ emerge --info python Portage 2.2.8-r1 (default/linux/x86/13.0/desktop/kde, gcc-4.7.3, glibc-2.17, 3.15.1 i686) ================================================================= System Settings ================================================================= System uname: Linux-3.15.1-i686-Intel-R-_Core-TM-_i5-2540M_CPU_@_2.60GHz-with-gentoo-2.2 KiB Mem: 8173096 total, 1874968 free KiB Swap: 9767516 total, 9765168 free Timestamp of tree: Tue, 17 Jun 2014 15:15:01 +0000 ld GNU ld (GNU Binutils) 2.23.2 ccache version 3.1.9 [disabled] app-shells/bash: 4.2_p45 dev-java/java-config: 2.2.0 dev-lang/python: 2.7.6, 3.2.5-r3, 3.3.3 dev-util/ccache: 3.1.9-r3 dev-util/cmake: 2.8.12.2 dev-util/pkgconfig: 0.28 sys-apps/baselayout: 2.2 sys-apps/openrc: 0.12.4 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.11.6, 1.12.6, 1.13.4 sys-devel/binutils: 2.23.2 sys-devel/gcc: 4.7.3-r1 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.13 (virtual/os-headers) sys-libs/glibc: 2.17 Repositories: gentoo toralf ACCEPT_KEYWORDS="x86" ACCEPT_LICENSE="*" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=native -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.5/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -march=native -pipe" DISTDIR="/var/lib/distfiles" EMERGE_DEFAULT_OPTS="--keep-going=y --nospinner --tree --quiet-build --deep" FCFLAGS="-O2 -march=i686 -pipe" FEATURES="assume-digests binpkg-logs compress-build-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -march=i686 -pipe" GENTOO_MIRRORS="http://ftp.uni-erlangen.de/pub/mirrors/gentoo http://mirror.qubenet.net/mirror/gentoo/ http://mirror.bytemark.co.uk/gentoo/ http://gd.tuwien.ac.at/opsys/linux/gentoo/" LANG="en_US.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j4" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.de.gentoo.org/gentoo-portage" USE="X a52 aac acl acpi alsa apache2 berkdb bittorrent branding bzip2 cairo cdda cdr classic cli consolekit corefonts cracklib crypt css cups cxx dbus declarative dri dts dvb dvd dvdr emboss encode exif fam fastbuild ffmpeg firefox flac fontconfig fortran g3dvl gdbm gif git gmp gnuplot gphoto2 gpm gtk hyphenation iconv ipv6 isag java jpeg kde kipi kqemu kvm lcms libnotify logrotate mad mbox mmx mng modules mp3 mp4 mpeg mysql mysqli ncurses nls nptl nsplugin ntfs ogg opengl openmp pam pango pcre pdf phonon pkcs11 plasma png policykit ppds qemu qt3support qt4 rdesktop readline sdl session sna spell spice sse sse2 sse4_1 ssl ssse3 startup-notification svg tcpd thinkpad threads tiff tk transparent-proxy truetype udev udisks uml unicode upower usb uxa v4l vaapi vim-syntax vnc vorbis wxwidgets x264 x86 xa xattr xcb xcomposite xinerama xml xpm xscreensaver xv xvid xvmc zlib" ABI_X86="32" ALSA_CARDS="hda-intel" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en en_GB" LIRC_DEVICES="devinput" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="intel" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON ================================================================= Package Settings ================================================================= dev-lang/python-2.7.6 was built with the following: USE="gdbm ipv6 ncurses (policykit) readline sqlite ssl threads tk (wide-unicode) xml -berkdb -build -doc -examples -hardened -wininst" CFLAGS="-O2 -march=native -pipe -fwrapv" CXXFLAGS="-O2 -march=native -pipe -fwrapv" LDFLAGS="-Wl,-O1 -Wl,--as-needed -L." dev-lang/python-3.2.5-r3 was built with the following: USE="gdbm ipv6 ncurses (policykit) readline sqlite ssl threads tk (wide-unicode) xml -build -doc -examples -hardened -wininst" CFLAGS="-O2 -march=native -pipe -fwrapv" CXXFLAGS="-O2 -march=native -pipe -fwrapv" LDFLAGS="-Wl,-O1 -Wl,--as-needed -L." dev-lang/python-3.3.3 was built with the following: USE="gdbm ipv6 ncurses (policykit) readline sqlite ssl threads tk xml -build -doc -examples -hardened -wininst" CFLAGS="-O2 -march=native -pipe -fwrapv" CXXFLAGS="-O2 -march=native -pipe -fwrapv" LDFLAGS="-Wl,-O1 -Wl,--as-needed -L."
This is not a bug; that file does not use UTF-8 encoding, but rather ISO-8859. % file drivers/staging/rtl8821ae/btcoexist/HalBtc8812a1Ant.c drivers/staging/rtl8821ae/btcoexist/HalBtc8812a1Ant.c: C source, ISO-8859 text, with CRLF, LF line terminators You need to specify the correct encoding as a argument to the open() function.
