Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 369913

Summary: sys-apps/portage- passes params as dict instead of string to http.client.HTTPConnection.request()
Product: Portage Development Reporter: Yun Zheng Hu <hu>
Component: CoreAssignee: Portage team <dev-portage>
Severity: normal Keywords: InVCS
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 358927    
Attachments: make_http_request: fix request arguments

Description Yun Zheng Hu 2011-06-03 14:53:34 UTC
I use an old style PORTAGE_BINHOST webserver, meaning it hosts raw *.tbz2 files instead of using a 'Packages' metadata file.

This still works fine with Python 2.6 + portage- But when I upgrade python to 2.7 it breaks. I have tracked it down to this function:

in the file /usr/lib/portage/pym/portage/

def make_http_request(conn, address, params={}, headers={}, dest=None):
                        conn.request("GET", address, params, headers)

In 2.7 httplib will throw the following error:

Traceback (most recent call last):
  File "/usr/bin/emerge", line 43, in <module>
    retval = emerge_main()
  File "/usr/lib/portage/pym/_emerge/", line 1437, in emerge_main
    getbinpkgs="--getbinpkg" in myopts)
  File "/usr/lib/portage/pym/portage/dbapi/", line 511, in populate
  File "/usr/lib/portage/pym/portage/dbapi/", line 924, in _populate
    base_url, chunk_size=chunk_size)
  File "/usr/lib/portage/pym/portage/", line 572, in dir_get_metadata
    filelist = dir_get_list(baseurl, conn)
  File "/usr/lib/portage/pym/portage/", line 360, in dir_get_list
    page,rc,msg = make_http_request(conn,address,params,headers)
  File "/usr/lib/portage/pym/portage/", line 273, in make_http_request
    raise e
TypeError: must be string or buffer, not dict

The TypeError refers to the params variable. 
The correct way to call the method is using urllib.urlencode on the params dict to encode it to a string, as seen on

For example:

import urllib
params = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})

Reproducible: Always

Steps to Reproduce:
1. Host a directory of *.tbz2 files on a webserver and use this as PORTAGE_BINHOST
2. Install Python 2.7 + portage-
3. Try emerging a binary package using: emerge -GK <package>
4. emerge will throw error: 

!!! Error connecting to '<url>'.
!!! Unable to get listing: None Server request failed: must be string or buffer, not dict

Actual Results:  
emerge could not find 'Packages' metafile and tries to fallback to directory listing but fails to do so.

Expected Results:  
emerge retrieving directory listing of PORTAGE_BINHOST even when there is no 'Packages' metadata file.

works with Python 2.6, breaks in Python 2.7+
Comment 1 Zac Medico gentoo-dev 2011-06-03 21:55:31 UTC
Created attachment 275745 [details, diff]
make_http_request: fix request arguments

Save as /tmp/http_request.patch and apply as follows:

  patch /usr/lib/portage/pym/portage/ /tmp/http_request.patch
Comment 2 Yun Zheng Hu 2011-06-06 12:13:42 UTC
(In reply to comment #1)
> Created attachment 275745 [details, diff]
> make_http_request: fix request arguments
> Save as /tmp/http_request.patch and apply as follows:
>   patch /usr/lib/portage/pym/portage/ /tmp/http_request.patch

Patch works, tested with Python 2.6.5 and 2.7.1
Comment 3 Zac Medico gentoo-dev 2011-06-06 13:02:25 UTC
Thanks for testing. This is fixed in 2.1.10 and 2.2.0_alpha38.