Bug 369913 – sys-apps/portage-2.1.9.42: getbinpkg.py passes params as dict instead of string to http.client.HTTPConnection.request()

• Status: RESOLVED FIXED
• Product: Portage Development

[Description Yun Zheng Hu 2011-06-03 14:53:34 UTC]

I use an old style PORTAGE_BINHOST webserver, meaning it hosts raw *.tbz2 files instead of using a 'Packages' metadata file.

This still works fine with Python 2.6 + portage-2.1.9.42. But when I upgrade python to 2.7 it breaks. I have tracked it down to this function:

in the file /usr/lib/portage/pym/portage/getbinpkg.py:

---
def make_http_request(conn, address, params={}, headers={}, dest=None):
...
                        conn.request("GET", address, params, headers)
---

In 2.7 httplib will throw the following error:

Traceback (most recent call last):
  File "/usr/bin/emerge", line 43, in <module>
    retval = emerge_main()
  File "/usr/lib/portage/pym/_emerge/main.py", line 1437, in emerge_main
    getbinpkgs="--getbinpkg" in myopts)
  File "/usr/lib/portage/pym/portage/dbapi/bintree.py", line 511, in populate
    self._populate(getbinpkgs)
  File "/usr/lib/portage/pym/portage/dbapi/bintree.py", line 924, in _populate
    base_url, chunk_size=chunk_size)
  File "/usr/lib/portage/pym/portage/getbinpkg.py", line 572, in dir_get_metadata
    filelist = dir_get_list(baseurl, conn)
  File "/usr/lib/portage/pym/portage/getbinpkg.py", line 360, in dir_get_list
    page,rc,msg = make_http_request(conn,address,params,headers)
  File "/usr/lib/portage/pym/portage/getbinpkg.py", line 273, in make_http_request
    raise e
TypeError: must be string or buffer, not dict


The TypeError refers to the params variable. 
The correct way to call the method is using urllib.urlencode on the params dict to encode it to a string, as seen on http://docs.python.org/library/httplib.html.

For example:

import urllib
params = urllib.parse.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})




Reproducible: Always

Steps to Reproduce:
1. Host a directory of *.tbz2 files on a webserver and use this as PORTAGE_BINHOST
2. Install Python 2.7 + portage-2.1.9.42
3. Try emerging a binary package using: emerge -GK <package>
4. emerge will throw error: 

!!! Error connecting to '<url>'.
!!! Unable to get listing: None Server request failed: must be string or buffer, not dict

Actual Results:  
emerge could not find 'Packages' metafile and tries to fallback to directory listing but fails to do so.

Expected Results:  
emerge retrieving directory listing of PORTAGE_BINHOST even when there is no 'Packages' metadata file.

works with Python 2.6, breaks in Python 2.7+

[Comment 1 Zac Medico 2011-06-03 21:55:31 UTC]

Created attachment 275745 [details, diff]
make_http_request: fix request arguments

Save as /tmp/http_request.patch and apply as follows:

  patch /usr/lib/portage/pym/portage/getbinpkg.py /tmp/http_request.patch

[Comment 2 Yun Zheng Hu 2011-06-06 12:13:42 UTC]

(In reply to comment #1)
> Created attachment 275745 [details, diff]
> make_http_request: fix request arguments
> 
> Save as /tmp/http_request.patch and apply as follows:
> 
>   patch /usr/lib/portage/pym/portage/getbinpkg.py /tmp/http_request.patch

Patch works, tested with Python 2.6.5 and 2.7.1

[Comment 3 Zac Medico 2011-06-06 13:02:25 UTC]

Thanks for testing. This is fixed in 2.1.10 and 2.2.0_alpha38.