Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 363897 - Cannot use BINHOST when system python set to python3
Summary: Cannot use BINHOST when system python set to python3
Status: RESOLVED OBSOLETE
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL: http://bugs.python.org/issue4608
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-04-17 02:28 UTC by Albert W. Hopkins
Modified: 2012-05-19 09:36 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Albert W. Hopkins 2011-04-17 02:28:33 UTC
I am trying to use a portage binhost with the latest stage3, where python3 is the default python.  However, I get the error:

Traceback (most recent call last):
  File "/usr/bin/emerge", line 43, in <module>
    retval = emerge_main()
  File "/usr/lib64/portage/pym/_emerge/main.py", line 1437, in emerge_main
    getbinpkgs="--getbinpkg" in myopts)
  File "/usr/lib64/portage/pym/portage/dbapi/bintree.py", line 511, in populate
    self._populate(getbinpkgs)
  File "/usr/lib64/portage/pym/portage/dbapi/bintree.py", line 816, in _populate
    rmt_idx.readHeader(f_dec)
  File "/usr/lib64/portage/pym/portage/getbinpkg.py", line 817, in readHeader
    self.header.update(self._readpkgindex(pkgfile, pkg_entry=False))
  File "/usr/lib64/portage/pym/portage/getbinpkg.py", line 789, in _readpkgindex
    for line in pkgfile:
  File "/usr/lib64/python3.1/codecs.py", line 1024, in iterdecode
    for input in iterator:
TypeError: 'addinfourl' object is not iterable

Fatal error: run() encountered an error (return code 1) while executing 'emerge -ngk =dev-db/postgresql-server-9.0*'

If I don't use "-ngk" the install goes fine (but without using the BINHOST).  If I use eselect to change the system python to python2.7 then the install goes fine (using the BINHOST).

So it appears to be an implementation difference with python3.


root@tp-db $ emerge --info
Portage 2.1.9.42 (default/linux/amd64/10.0, gcc-4.4.5, glibc-2.11.3-r0, 2.6.36-gentoo-r8 x86_64)
=================================================================
System uname: Linux-2.6.36-gentoo-r8-x86_64-QEMU_Virtual_CPU_version_0.13.0-with-gentoo-1.12.14
Timestamp of tree: Sat, 16 Apr 2011 00:45:01 +0000
app-shells/bash:     4.1_p9
dev-lang/python:     2.7.1-r1, 3.1.3-r1
sys-apps/baselayout: 1.12.14-r1
sys-apps/sandbox:    2.4
sys-devel/autoconf:  2.65-r1
sys-devel/automake:  1.11.1
sys-devel/binutils:  2.20.1-r1
sys-devel/gcc:       4.4.5
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.10
sys-devel/make:      3.81-r2
sys-kernel/linux-headers: 2.6.36.1
virtual/os-headers:  0
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-Os -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/eselect/postgresql /etc/gconf /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-Os -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--jobs=2"
FEATURES="assume-digests binpkg-logs buildpkg distlocks fixlafiles fixpackages news nodoc noinfo noman parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS=""
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j2"
PKGDIR="/var/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="amd64 multilib nptl pam unicode" ALSA_PCM_PLUGINS="*" ELIBC="glibc" KERNEL="linux" USERLAND="GNU" 
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY
Comment 1 Zac Medico gentoo-dev 2011-04-18 04:53:36 UTC
Which protocol are you using for PORTAGE_BINHOST?
Comment 2 Albert W. Hopkins 2011-04-18 11:01:56 UTC
(In reply to comment #1)
> Which protocol are you using for PORTAGE_BINHOST?

Using FTP.
Comment 3 Zac Medico gentoo-dev 2011-04-18 20:45:01 UTC
The function says it's supposed to return a file-like object:

http://docs.python.org/dev/library/urllib.request.html#urllib.request.urlopen

Since file-like objects typically support iteration, it seems odd that this one doesn't. I'll setup an ftp server and see if I can reproduce it. Given issues like bug 350139, we should consider making it always use FETCHCOMMAND instead of urlopen for the ftp protocol.
Comment 4 Albert W. Hopkins 2011-04-18 23:37:32 UTC
Found this python bug.  It's fixed, but perhaps not in Python 3.1.3?

http://bugs.python.org/issue4608
Comment 5 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2011-04-18 23:40:22 UTC
Could you test with Python 3.2?
(Python 3.2 will be unmasked probably in next month.)
Comment 6 Albert W. Hopkins 2011-04-20 07:26:47 UTC
(In reply to comment #5)
> Could you test with Python 3.2?
> (Python 3.2 will be unmasked probably in next month.)

Seems to still happen with Python 3.2:

root@tp-db $ python --version
Python 3.2

root@tp-db $ emerge -ngk postgresql-server
Traceback (most recent call last):
  File "/usr/bin/emerge", line 43, in <module>
    retval = emerge_main()
  File "/usr/lib64/portage/pym/_emerge/main.py", line 1437, in emerge_main
    getbinpkgs="--getbinpkg" in myopts)
  File "/usr/lib64/portage/pym/portage/dbapi/bintree.py", line 511, in populate
    self._populate(getbinpkgs)
  File "/usr/lib64/portage/pym/portage/dbapi/bintree.py", line 816, in _populate
    rmt_idx.readHeader(f_dec)
  File "/usr/lib64/portage/pym/portage/getbinpkg.py", line 817, in readHeader
    self.header.update(self._readpkgindex(pkgfile, pkg_entry=False))
  File "/usr/lib64/portage/pym/portage/getbinpkg.py", line 789, in _readpkgindex
    for line in pkgfile:
  File "/usr/lib64/python3.2/codecs.py", line 1027, in iterdecode
    for input in iterator:
TypeError: 'addinfourl' object is not iterable

root@tp-db $ grep BINHOST /etc/make.conf
PORTAGE_BINHOST=ftp://192.168.122.1/vm-packages/
Comment 7 Albert W. Hopkins 2011-04-20 07:35:33 UTC
Perhaps it is not the python bug previously referenced.  As the test case from that bug passes:


root@tp-db $ cat test.py
from urllib.request import urlopen
for line in urlopen('http://python.org/'):
        print(line)
        break

root@tp-db $ python3.2 test.py
b'<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\n'

However:

root@tp-db $ python3.2 /usr/bin/emerge -ngk postgresql-server
Traceback (most recent call last):
  File "/usr/bin/emerge", line 43, in <module>
    retval = emerge_main()
  File "/usr/lib64/portage/pym/_emerge/main.py", line 1437, in emerge_main
    getbinpkgs="--getbinpkg" in myopts)
  File "/usr/lib64/portage/pym/portage/dbapi/bintree.py", line 511, in populate
    self._populate(getbinpkgs)
  File "/usr/lib64/portage/pym/portage/dbapi/bintree.py", line 816, in _populate
    rmt_idx.readHeader(f_dec)
  File "/usr/lib64/portage/pym/portage/getbinpkg.py", line 817, in readHeader
    self.header.update(self._readpkgindex(pkgfile, pkg_entry=False))
  File "/usr/lib64/portage/pym/portage/getbinpkg.py", line 789, in _readpkgindex
    for line in pkgfile:
  File "/usr/lib64/python3.2/codecs.py", line 1027, in iterdecode
    for input in iterator:
TypeError: 'addinfourl' object is not iterable
Comment 8 Zac Medico gentoo-dev 2011-04-20 07:40:07 UTC
(In reply to comment #7)
> root@tp-db $ cat test.py
> from urllib.request import urlopen
> for line in urlopen('http://python.org/'):
>         print(line)
>         break

Modify the test case to point at $PORTAGE_BINHOST/Packages instead of http://python.org/.
Comment 9 Albert W. Hopkins 2011-04-20 07:43:09 UTC
Oh, apparently it does not like the FTP protocol:



root@tp-db $ cat test.py
from urllib.request import urlopen
for line in urlopen('ftp://gentoo.osuosl.org/pub/gentoo/releases/'):
        print(line)
        break

root@tp-db $ python3.2 test.py
Traceback (most recent call last):
  File "test.py", line 2, in <module>
    for line in urlopen('ftp://gentoo.osuosl.org/pub/gentoo/releases/'):
TypeError: 'addinfourl' object is not iterable
Comment 10 Albert W. Hopkins 2011-04-20 07:47:45 UTC
(In reply to comment #8)
> (In reply to comment #7)
> Modify the test case to point at $PORTAGE_BINHOST/Packages instead of
> http://python.org/.

root@tp-db $ cat test.py
from urllib.request import urlopen
for line in urlopen('ftp://192.168.122.1/vm-packages/Packages'):
        print(line)
        break
 
root@tp-db $ python3.2 test.py
Traceback (most recent call last):
  File "test.py", line 2, in <module>
    for line in urlopen('ftp://192.168.122.1/vm-packages/Packages'):
TypeError: 'addinfourl' object is not iterable
Comment 11 Albert W. Hopkins 2011-04-20 10:47:33 UTC
The Python bug is re-opened.
Comment 12 Zac Medico gentoo-dev 2011-12-29 03:10:48 UTC
Maybe this is fixed in Python 3.2.2.
Comment 13 Albert W. Hopkins 2012-04-26 13:12:24 UTC
(In reply to comment #12)
> Maybe this is fixed in Python 3.2.2.

The code in comment #9 now works with 3.2.2.  I haven't tested the BINHOST yet, but will hopefully in the next day or so.
Comment 14 Albert W. Hopkins 2012-05-19 09:36:57 UTC
Works with 3.2.x now.  Thanks.