Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 686450 - app-portage/unsymlink-lib-13 crash while scanning a directory with utf-8 file name
Summary: app-portage/unsymlink-lib-13 crash while scanning a directory with utf-8 file...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Michał Górny
URL:
Whiteboard:
Keywords:
: 687478 687718 (view as bug list)
Depends on:
Blocks:
 
Reported: 2019-05-21 08:43 UTC by Tupone Alfredo
Modified: 2020-02-11 22:40 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tupone Alfredo gentoo-dev 2019-05-21 08:43:15 UTC
# unsymlink-lib --analyze raises UnicodeEncodeError
The error messages are:
Analyzing files installed into lib & lib64...
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.6/unsymlink-lib", line 778, in <module>
    main()
  File "/usr/lib/python-exec/python3.6/unsymlink-lib", line 669, in main
    m.analyze(usr_merge, prefixes)
  File "/usr/lib/python-exec/python3.6/unsymlink-lib", line 170, in analyze
    if not os.path.exists(f):
  File "/usr/lib64/python3.6/genericpath.py", line 19, in exists
    os.stat(path)
UnicodeEncodeError: 'ascii' codec can't encode character '\xc4' in position 42: ordinal not in range(128)

The last directory scanned is 
/usr/lib/go/test/fixedbugs/issue27836.dir/
from dev-lang/go

Reproducible: Always




The system uses python-3.6
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2019-05-21 09:00:16 UTC
What's your `locale`?
Comment 2 Tupone Alfredo gentoo-dev 2019-05-21 10:19:34 UTC
(In reply to Michał Górny from comment #1)
> What's your `locale`?

tupone ~ # locale
LANG=C
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL=
Comment 3 Tupone Alfredo gentoo-dev 2019-05-21 10:21:47 UTC
changing locale to 

  [3]   en_US * 

make it works
Comment 4 Tupone Alfredo gentoo-dev 2019-05-21 10:26:17 UTC
but I got this warning

One or more package files are missing from the system. This should not
cause any problems but you may want to reinstall the packages
that installed them. The missing files are:
 
        /usr/lib/go/test/fixedbugs/issue27836.dir/foo.go
        /usr/lib/go/test/fixedbugs/issue27836.dir/main.go
        /usr/lib64/gio/modules/giomodule.cache

Maybe the last is not related but the first two are in the directory with utf 8 names
Comment 5 Tupone Alfredo gentoo-dev 2019-05-22 09:43:01 UTC
app-portage/unsymlink-lib-9999 works to me
Comment 6 Larry the Git Cow gentoo-dev 2019-05-23 14:32:16 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=eb2551581927e698ce7ac7ff538a15c4360909e6

commit eb2551581927e698ce7ac7ff538a15c4360909e6
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: 2019-05-23 14:29:30 +0000
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2019-05-23 14:32:05 +0000

    app-portage/unsymlink-lib: Bump to v14
    
    Closes: https://bugs.gentoo.org/686450
    Signed-off-by: Michał Górny <mgorny@gentoo.org>

 app-portage/unsymlink-lib/Manifest                |  1 +
 app-portage/unsymlink-lib/unsymlink-lib-14.ebuild | 30 +++++++++++++++++++++++
 2 files changed, 31 insertions(+)
Comment 7 Mike Gilbert gentoo-dev 2019-06-07 17:50:29 UTC
*** Bug 687478 has been marked as a duplicate of this bug. ***
Comment 8 Brian Evans (RETIRED) gentoo-dev 2019-06-09 14:00:44 UTC
*** Bug 687718 has been marked as a duplicate of this bug. ***
Comment 9 Roger 2020-02-11 22:40:25 UTC
=app-portage/unsymlink-lib-17-r1 here.

I just upgraded from 17.0 to 17.1 and ran into this same identical bug.



rm -d /usr/lib64/go/test/fixedbugs/issue4964.dir/a.go
rm -d /usr/lib64/go/test/fixedbugs/issue4964.dir/b.go
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.6/unsymlink-lib", line 827, in <module>
    main()
  File "/usr/lib/python-exec/python3.6/unsymlink-lib", line 790, in main
    m.finish(pretend=args.pretend, resume=args.action.endswith('resume'))
  File "/usr/lib/python-exec/python3.6/unsymlink-lib", line 592, in finish
    out('rm -d {}', fp)
  File "/usr/lib/python-exec/python3.6/unsymlink-lib", line 41, in out
    _log(template, *args, **kwargs)
  File "/usr/lib/python-exec/python3.6/unsymlink-lib", line 30, in _log
    **kwargs
UnicodeEncodeError: 'ascii' codec can't encode character '\xc4' in position 50: ordinal not in range(128)



Performing either of the following seems to have temporarily resolved the problem.

# LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 unsymlink

or 

File /etc/env.d/02locale
LANG="en_US.UTF-8"
LC_COLLATE="C"

# env-update && source /etc/profile


I, as likely many others, dislike typing/reading non-ascii chars on their systems.

My initial search seems to have found this is a common bug with Python scripting:

"UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 20: ordinal not in range(128)"
https://stackoverflow.com/questions/9942594/unicodeencodeerror-ascii-codec-cant-encode-character-u-xa0-in-position-20

Suggest re-opening.