Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 610016 - app-portage/mirrorselect python3 incompatibility - TypeError: sequence item 1: expected str instance, bytes found
Summary: app-portage/mirrorselect python3 incompatibility - TypeError: sequence item 1...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage Tools Team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks:
 
Reported: 2017-02-19 14:30 UTC by Justin Lecher (RETIRED)
Modified: 2017-09-25 13:24 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
map hosts to str (mirrorselect.fix.patch,780 bytes, patch)
2017-02-19 19:03 UTC, zakalwe
Details | Diff
another attempt (mirrorselect.fix.patch,489 bytes, patch)
2017-02-20 19:42 UTC, zakalwe
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Justin Lecher (RETIRED) gentoo-dev 2017-02-19 14:30:03 UTC
Running 

mirrorselect -s3 -b10 -o

With

$ (1) eselect python list
Available Python interpreters, in order of preference:
  [1]   python3.5
  [2]   python3.4 (fallback)
  [3]   python2.7 (fallback)
  [4]   pypy3 (fallback)
  [5]   pypy (fallback)

results in

Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.4/mirrorselect", line 61, in <module>
    MirrorSelect().main(sys.argv)
  File "/usr/lib64/python3.4/site-packages/mirrorselect/main.py", line 379, in main
    config_path, options.rsync)
  File "/usr/lib64/python3.4/site-packages/mirrorselect/main.py", line 117, in change_config
    mirror_string = '%s="%s"' % (var, ' '.join(hosts))
TypeError: sequence item 1: expected str instance, bytes found
Comment 1 zakalwe 2017-02-19 19:03:26 UTC
Created attachment 464362 [details, diff]
map hosts to str

This patch might help.  I can't see exactly what the problem is here as it works ok on my machine.  If the output is garbled I will look further.
Comment 2 Arfrever Frehtes Taifersar Arahesis 2017-02-20 14:35:51 UTC
Comment on attachment 464362 [details, diff]
map hosts to str

This patch is obviously incorrect and would result in 'BytesWarning: str() on a bytes instance'.

As you can see below, str(bytes) results in b' prefix and ' suffix in resulting string:

$ python -bq
>>> str(b"a")
__main__:1: BytesWarning: str() on a bytes instance
"b'a'"
>>> b"a".decode("utf-8")
'a'
>>>
Comment 3 zakalwe 2017-02-20 19:42:22 UTC
Created attachment 464458 [details, diff]
another attempt

Ah sorry.  I'm guessing the problem is only the first host is checked if it is encoded.
Comment 4 Arfrever Frehtes Taifersar Arahesis 2017-02-20 21:47:32 UTC
if hasattr(hosts[i], 'decode'):
should be
if isinstance(hosts[i], bytes):
Comment 5 Brian Dolbec (RETIRED) gentoo-dev 2017-02-21 03:21:43 UTC
(In reply to zakalwe from comment #3)
> Created attachment 464458 [details, diff] [details, diff]
> another attempt
> 
> Ah sorry.  I'm guessing the problem is only the first host is checked if it
> is encoded.

Yeah, In all my testing, if it was either ascii or bytes for all entries.

I'm wondering if either someone pushed some ascii into the list of more probable, the first entry was a null string.

Thanks, no need to make a new patch, I'll change it to use isinstance() instead.

I was thinking this morning I might do it in a one-line generator, but this is clearer.

Thank you.

Pushed in commit: https://gitweb.gentoo.org/proj/mirrorselect.git/commit/?id=c91509d823c27fdb1527c61f77c959b305170915
Comment 7 Brian Dolbec (RETIRED) gentoo-dev 2017-02-21 07:20:59 UTC
Thanks Zac.  That was a Duh moment...  I blame working on unit tests all day where I was having to put nearly everything in quotes.  I must have hit undo in the editor without noticing I undid that fix.
Comment 8 Pacho Ramos gentoo-dev 2017-04-22 09:47:34 UTC
By the way, when committing this fix, it would be nice to add support for python3.5 and 3.6 if possible ;)

Thanks
Comment 9 Pacho Ramos gentoo-dev 2017-06-22 09:40:34 UTC
Is this a matter of preparing a new release with this and other fixes or applying the patches would be enough? Thanks :)
Comment 10 Brian Dolbec (RETIRED) gentoo-dev 2017-06-22 14:19:58 UTC
yes, I'll get a release out this weekend.  Thanks for the reminder...
Comment 11 Pacho Ramos gentoo-dev 2017-09-25 13:24:32 UTC
2.2.3 was released