Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 280259

Summary: sys-apps/portage-2.2_rc35 UnicodeDecodeError with --buildpkg or quickpkg
Product: Gentoo Linux Reporter: Adrian Bassett <Adrian.Bassett>
Component: [OLD] Core systemAssignee: Portage team <dev-portage>
Status: RESOLVED FIXED    
Severity: major CC: 4glitch, dimanish, esigra, remi
Priority: High Keywords: InVCS, REGRESSION
Version: unspecified   
Hardware: x86   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: Fix the UnicodeDecodeError
Fix the UnicodeDecodeError

Description Adrian Bassett 2009-08-04 06:18:23 UTC
sys-apps/portage-2.2_rc35 was installed as an upgrade to sys-apps/portage-2.2_rc33.

After that emerge fails with following output:

# emerge -ND world
>>> Verifying ebuild manifests
>>> Starting parallel fetch
>>> Emerging (1 of 9) sys-apps/portage-2.2_rc35
>>> Installing (1 of 9) sys-apps/portage-2.2_rc35
*** Resuming merge...
>>> Verifying ebuild manifests
>>> Starting parallel fetch
>>> Emerging (1 of 8) app-portage/mirrorselect-2.0.0
Traceback (most recent call last):
  File "/usr/bin/emerge", line 40, in <module>
    retval = emerge_main()
  File "/usr/lib/portage/pym/_emerge/main.py", line 1391, in emerge_main
    myopts, myaction, myfiles, spinner)
  File "/usr/lib/portage/pym/_emerge/actions.py", line 446, in action_build
    retval = mergetask.merge()
  File "/usr/lib/portage/pym/_emerge/Scheduler.py", line 814, in merge
    rval = self._merge()
  File "/usr/lib/portage/pym/_emerge/Scheduler.py", line 1117, in _merge
    self._main_loop()
  File "/usr/lib/portage/pym/_emerge/Scheduler.py", line 1247, in _main_loop
    self._poll_loop()
  File "/usr/lib/portage/pym/_emerge/PollScheduler.py", line 127, in _poll_loop
    handler(f, event)
  File "/usr/lib/portage/pym/_emerge/SpawnProcess.py", line 197, in _output_handler
    self._unregister_if_appropriate(event)
  File "/usr/lib/portage/pym/_emerge/AbstractPollTask.py", line 27, in _unregister_if_appropriate
    self.wait()
  File "/usr/lib/portage/pym/_emerge/AsynchronousTask.py", line 41, in wait
    self._wait()
  File "/usr/lib/portage/pym/_emerge/SubProcess.py", line 77, in _wait
    self._set_returncode(wait_retval)
  File "/usr/lib/portage/pym/_emerge/EbuildBinpkg.py", line 43, in _set_returncode
    bintree.inject(pkg.cpv, filename=binpkg_tmpfile)
  File "/usr/lib/portage/pym/portage/dbapi/bintree.py", line 893, in inject
    pkgindex.write(f)
  File "/usr/lib/portage/pym/portage/getbinpkg.py", line 811, in write
    [(k, metadata[k]) for k in keys if metadata[k]])
  File "/usr/lib/portage/pym/portage/getbinpkg.py", line 760, in _writepkgindex
    (self._write_translation_map.get(k, k), v))
  File "/usr/lib/python2.6/codecs.py", line 686, in write
    return self.writer.write(data)
  File "/usr/lib/python2.6/codecs.py", line 351, in write
    data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 16: ordinal not in range(128)

Re-emerging rc33 also fails, even with -k emerge option.

System now appears broken.  Any ideas?

Reproducible: Always
Comment 1 Adrian Bassett 2009-08-04 06:38:41 UTC
Ok, it has been possible to re-install the rc33 version manually using ebuild and appropiate arguments so the system is not so broken.

It appears that the problem with rc35 is in the qmerge stage.

Incidentally, although I have to unmask portage in /etc/portage/package.unmask in order to use/test portage 2.2, it seems that masking a particular version of 2.2 in /etc/portage/package.mask is ignored so that the supposedly masked version is installed anyway.  This can't be intentional?
Comment 2 Zac Medico gentoo-dev 2009-08-04 06:51:00 UTC
(In reply to comment #1)
> Incidentally, although I have to unmask portage in /etc/portage/package.unmask
> in order to use/test portage 2.2, it seems that masking a particular version of
> 2.2 in /etc/portage/package.mask is ignored so that the supposedly masked
> version is installed anyway.  This can't be intentional?

It's intentional because you probably have a package.unmask entry which negates your package.mask entry. In order to avoid that, use a more specific package.unmask entry, such as =sys-apps/portage-2.2_rc33. Alternatively, you can use -atom to revert atom in package.mask (instead of using package.unmask).
Comment 3 Zac Medico gentoo-dev 2009-08-04 06:52:21 UTC
Created attachment 200116 [details, diff]
Fix the UnicodeDecodeError

If this patch is saved as /tmp/unicode_packages.patch, then it can be applied as follows:

  patch /usr/lib/portage/pym/portage/dbapi/bintree.py /tmp/unicode_packages.patch
Comment 4 Zac Medico gentoo-dev 2009-08-04 07:41:37 UTC
Created attachment 200120 [details, diff]
Fix the UnicodeDecodeError

This patch handles more cases that the previous on. If it is saved as /tmp/unicode_packages.patch, then it can be applied as follows:

  cd /usr/lib/portage
  patch -p0 < /tmp/unicode_packages.patch
Comment 5 Rémi Cardona (RETIRED) gentoo-dev 2009-08-04 10:42:17 UTC
confirming the bug with a plain old update of mirrorselect (no buildpkg foo).

Thanks
Comment 6 Rémi Cardona (RETIRED) gentoo-dev 2009-08-04 11:51:51 UTC
This patch doesn't work for me. Same error. (unless something needs to be "recompiled" after patching?"

Cheers
Comment 7 Zac Medico gentoo-dev 2009-08-04 18:52:08 UTC
(In reply to comment #6)
> This patch doesn't work for me. Same error. (unless something needs to be
> "recompiled" after patching?"

It's probably a slightly different UnicodeDecodeError. It's common for people to say 'Same error' when it's really not identical in cases like this.
Comment 8 Zac Medico gentoo-dev 2009-08-05 01:19:45 UTC
This is fixed in 2.2_rc36.
Comment 9 Rémi Cardona (RETIRED) gentoo-dev 2009-08-05 09:27:46 UTC
@Zac, how do I upgrade to make sure the bug is fixed?

Thanks
Comment 10 Zac Medico gentoo-dev 2009-08-05 09:58:24 UTC
(In reply to comment #9)
> @Zac, how do I upgrade to make sure the bug is fixed?

wget http://distfiles.gentoo.org/distfiles/portage-2.2_rc36.patch.bz2 -O \ /tmp/portage-2.2_rc36.patch.bz2
bunzip2 /tmp/portage-2.2_rc36.patch.bz2
cd /usr/lib/portage
patch -p0 < /tmp/portage-2.2_rc36.patch

The ChangeLog part of the patch won't apply, but everything else should.
Comment 11 Adrian Bassett 2009-08-05 10:19:24 UTC
Thanks for the clarifications in comment #2.

I was away soon after reporting the problem so was not able to test the individual patches before rc36 was released.  However, I have now installed that version and 'emerge -ND world' finished without any (portage) problem so things seem fixed.

Many thanks.
 

Comment 12 Rémi Cardona (RETIRED) gentoo-dev 2009-08-05 14:13:24 UTC
(In reply to comment #10)
> The ChangeLog part of the patch won't apply, but everything else should.

More than half of the patch didn't apply cleanly. wired gave me a binpkg of rc33 I brutally untared in /. I'll see if rc36 works.

Thanks :)
Comment 13 oleg 2009-08-08 08:38:13 UTC
(In reply to comment #10)
> (In reply to comment #9)
> > @Zac, how do I upgrade to make sure the bug is fixed?
> 
> wget http://distfiles.gentoo.org/distfiles/portage-2.2_rc36.patch.bz2 -O \
> /tmp/portage-2.2_rc36.patch.bz2
> bunzip2 /tmp/portage-2.2_rc36.patch.bz2
> cd /usr/lib/portage
> patch -p0 < /tmp/portage-2.2_rc36.patch
> 
> Trying to emerge portage-2.2_rc36 giving an UnicodeDecodeError, however , applying  a patch as described works.

Comment 14 Dmitri Bogomolov 2009-08-26 07:35:36 UTC
in sys-apps/portage-2.2_rc38:

emerge --sync


Performing Global Updates: /usr/portage/profiles/updates/3Q-2009
(Could take a couple of minutes if you have a lot of binary packages.)
  .='update pass'  *='binary update'  #='/var/db update'  @='/var/db move'
  s='/var/db SLOT move'  %='binary move'  S='binary SLOT move'
  p='update /etc/portage/package.*'
.........................
pppppTraceback (most recent call last):
  File "/usr/bin/emerge", line 40, in <module>
    retval = emerge_main()
  File "//usr/lib/portage/pym/_emerge/main.py", line 1034, in emerge_main
    if portage._global_updates(trees, mtimedb["updates"]):
  File "//usr/lib/portage/pym/portage/__init__.py", line 8193, in _global_updates
    moves = bindb.move_ent(update_cmd)
  File "//usr/lib/portage/pym/portage/dbapi/bintree.py", line 277, in move_ent
    mytbz2.recompose_mem(portage.xpak.xpak_mem(mydata))
  File "//usr/lib/portage/pym/portage/xpak.py", line 94, in xpak_mem
    dataglob=dataglob+newglob
UnicodeDecodeError: 'ascii' codec can't decode byte 0xb4 in position 10: ordinal not in range(128)
Comment 15 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2009-08-26 10:34:34 UTC
(In reply to comment #14)

Bug #282505.