Summary: | sys-apps/portage - app-misc/ca-certificates-20230311.3.90 UnicodeEncodeError: 'latin-1' codec can't encode character '\u0151 | ||
---|---|---|---|
Product: | Portage Development | Reporter: | Pietro <pietro.sammarco> |
Component: | Core | Assignee: | Portage team <dev-portage> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | base-system, egger.m, jstein, sam |
Priority: | Normal | Keywords: | InVCS |
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
See Also: |
https://bugs.gentoo.org/show_bug.cgi?id=876193 https://bugs.gentoo.org/show_bug.cgi?id=722270 https://github.com/gentoo/portage/pull/1086 |
||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | 912676 | ||
Bug Blocks: | |||
Attachments: |
ca-certificates-20230311.3.90:20230827-103750.log
emerge --info make.conf /etc/portage/env/nolto.conf /etc/portage/package.env/noltobuild |
Description
Pietro
2023-08-27 10:50:59 UTC
Created attachment 868808 [details]
ca-certificates-20230311.3.90:20230827-103750.log
Created attachment 868816 [details]
emerge --info
Created attachment 868817 [details]
make.conf
Created attachment 868818 [details]
/etc/portage/env/nolto.conf
Created attachment 868819 [details]
/etc/portage/package.env/noltobuild
Update: I renamed /usr/share/ca-certificates/mozilla/NetLock_Arany_\=Class_Gold\=_Főtanúsítvány.crt symlinked it to /etc/ssl/certs/._cfg0000_988a38cb.0 and ran update-ca-certificates I was the able to rebuild ca-certificates again. However, there seem to be few others certs whose name contain non-standard characters in /usr/share/ca-certificates/mozilla . While I think this issue can be close, I would like to figure out why these mozilla certs are named like that. Btw, the issue is not resolved. Everytime I rebuild ca-certificates it brings back a certificate containing a weird character in its name. The certificate in question is: '/usr/share/ca-certificates/mozilla/NetLock_Arany_=Class_Gold=_F�'$'\221''tanúsítvány.crt' Somebody in IRC suggested it may be a charset issue, but I do believe I have the charset set correctly, and have had this rootfs for the past 5 years and never experienced this before. Locales info below: >>> pietro@toilet ~ $ locale LANG=en_GB LC_CTYPE="en_GB" LC_NUMERIC="en_GB" LC_TIME="en_GB" LC_COLLATE=C.UTF-8 LC_MONETARY="en_GB" LC_MESSAGES="en_GB" LC_PAPER="en_GB" LC_NAME="en_GB" LC_ADDRESS="en_GB" LC_TELEPHONE="en_GB" LC_MEASUREMENT="en_GB" LC_IDENTIFICATION="en_GB" LC_ALL= pietro@toilet ~ $ <<< >>> pietro@toilet ~ $ cat /etc/locale.gen en_GB ISO-8859-1 en_GB.UTF-8 UTF-8 pietro@toilet ~ $ <<< >>> pietro@toilet ~ $ eselect locale show LANG variable in profile: en_GB.utf8 pietro@toilet ~ $ <<< Note, I am in the UK and my laptop has a GB keyboard. Thanks The error is triggered by setting LANG="en_GB". You can avoid it by setting LANG="en_GB.UTF-8" instead. This is likely a bug in Portage. Probably a regression introduced in the commit below. https://gitweb.gentoo.org/proj/portage.git/commit/?id=a87be47f7d3245050da43d7c3ab4760d47e9fac5 We need to ensure that file paths are always UTF-8 encoded (_encodings["merge"]). Could also be this change. https://gitweb.gentoo.org/proj/portage.git/commit/?id=5572b73744121f67c4e55040966bfe91a0e5fb6c _cmpxattr() needs to ensure merge encoding (UTF-8). (In reply to Pietro from comment #7) > Locales info below: > >>> > pietro@toilet ~ $ locale > LANG=en_GB > LC_CTYPE="en_GB" > LC_NUMERIC="en_GB" > LC_TIME="en_GB" > LC_COLLATE=C.UTF-8 > LC_MONETARY="en_GB" > LC_MESSAGES="en_GB" > LC_PAPER="en_GB" > LC_NAME="en_GB" > LC_ADDRESS="en_GB" > LC_TELEPHONE="en_GB" > LC_MEASUREMENT="en_GB" > LC_IDENTIFICATION="en_GB" > LC_ALL= > pietro@toilet ~ $ > <<< > > >>> > pietro@toilet ~ $ cat /etc/locale.gen > en_GB ISO-8859-1 > en_GB.UTF-8 UTF-8 > pietro@toilet ~ $ > <<< > > >>> > pietro@toilet ~ $ eselect locale show > LANG variable in profile: > en_GB.utf8 The "eselect profile" setting (en_GB.utf8) is being overridden with en_GB (no UTF-8) somewhere, likely in your shell startup files. You should really fix that. Hi Mike, I have /etc/locale.conf LANG="en_GB.UTF-8" LC_COLLATE="C.UTF-8" and >>> pietro@toilet ~ $ eselect locale show LANG variable in profile: en_GB.UTF-8 pietro@toilet ~ $ <<< However, the locale command still shows LANG=en_GB. >>> LANG=en_GB LC_CTYPE="en_GB" LC_NUMERIC="en_GB" LC_TIME="en_GB" LC_COLLATE=C.UTF-8 LC_MONETARY="en_GB" LC_MESSAGES="en_GB" LC_PAPER="en_GB" LC_NAME="en_GB" LC_ADDRESS="en_GB" LC_TELEPHONE="en_GB" LC_MEASUREMENT="en_GB" LC_IDENTIFICATION="en_GB" LC_ALL= <<< Thanks Btw, I really do not where this LAN value replacement script is coming from.
I do not have no startup script, and I start Plasma with sddm.
Can sddm be doing this?
I also found this environment.d script laying around which exports LANG, but it seems to have the correct value anyways.
>>>
pietro@toilet ~ $ cat /etc/environment.d/10-gentoo-env.conf
# THIS FILE IS AUTOMATICALLY GENERATED BY env-update.
# DO NOT EDIT THIS FILE.
ANDROID_HOME=/opt/android-sdk-update-manager
ANDROID_SWT=/usr/share/swt-3.7/lib
ANT_HOME=/usr/share/ant
CONFIG_PROTECT=/usr/share/gnupg/qualified.txt /usr/share/config
CONFIG_PROTECT_MASK=/etc/sandbox.d /etc/fonts/fonts.conf /etc/gentoo-release /etc/dconf /etc/ca-certificates.conf /etc/revdep-rebuild
EDITOR=vi
GSETTINGS_BACKEND=dconf
INFOPATH=/usr/share/gcc-data/x86_64-pc-linux-gnu/12/info:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.40/info:/usr/share/autoconf-2.71/info:/usr/share/automake-1.16.5/info:/usr/share/info
LANG=en_GB.utf8
LC_COLLATE=C.UTF-8
LESS=-R -M --shift 5
LESSOPEN=|lesspipe %s
LEX=flex
MANPAGER=manpager
MANPATH=/usr/share/gcc-data/x86_64-pc-linux-gnu/12/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.40/man:/etc/java-config-2/current-system-vm/man/:/usr/local/share/man:/usr/share/man:/usr/lib/rust/man:/usr/lib/llvm/16/share/man
PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/16/bin:/opt/android-sdk-update-manager/tools:/opt/android-sdk-update-manager/platform-tools
ROOTPATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/16/bin
XDG_CONFIG_DIRS=/etc/xdg
XDG_DATA_DIRS=/usr/local/share:/usr/share
pietro@toilet ~ $
<<<
Right, in tty I get the expected LANG charset. I started kde manually this time but still I got LANG=en_GB. I think this may have something to do with KDE itself. Found the culprit. .config/plasma-localerc was set to LANG=en_GB. Changed to en_GB.utf8, rebooted and everything works as expected. What a ride. Thanks (In reply to Mike Gilbert from comment #11) > Could also be this change. > > https://gitweb.gentoo.org/proj/portage.git/commit/ > ?id=5572b73744121f67c4e55040966bfe91a0e5fb6c > > _cmpxattr() needs to ensure merge encoding (UTF-8). Is is safe to assume UTF-8 encoding regardless of the selected system locale? I can take care of this. (In reply to Michael Egger from comment #17) > Is is safe to assume UTF-8 encoding regardless of the selected system locale? > I can take care of this. Yes, Portage is specifically coded to ignore the user's selected locale when dealing with file names. It always uses UTF-8. Tackled this in https://github.com/gentoo/portage/pull/1086. The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=1d3b22621d332331da6e48ae653e983406f44e5f commit 1d3b22621d332331da6e48ae653e983406f44e5f Author: gcarq <egger.m@protonmail.com> AuthorDate: 2023-09-01 16:46:23 +0000 Commit: Mike Gilbert <floppym@gentoo.org> CommitDate: 2023-09-01 19:13:27 +0000 emerge: ensure paths are UTF-8 encoded in _needs_move() Bug: https://bugs.gentoo.org/913103 Closes: https://github.com/gentoo/portage/pull/1086 Signed-off-by: Michael Egger <egger.m@protonmail.com> Signed-off-by: Mike Gilbert <floppym@gentoo.org> NEWS | 2 ++ lib/portage/dbapi/vartree.py | 7 +++++-- lib/portage/util/movefile.py | 5 +++-- 3 files changed, 10 insertions(+), 4 deletions(-) The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9f9211a35abef13079f93a96f57f3a96083c69a7 commit 9f9211a35abef13079f93a96f57f3a96083c69a7 Author: Sam James <sam@gentoo.org> AuthorDate: 2023-10-03 15:38:28 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-10-03 15:38:41 +0000 sys-apps/portage: add 3.0.52 Closes: https://bugs.gentoo.org/704866 Closes: https://bugs.gentoo.org/877793 Closes: https://bugs.gentoo.org/889300 Closes: https://bugs.gentoo.org/900224 Closes: https://bugs.gentoo.org/912676 Closes: https://bugs.gentoo.org/912808 Closes: https://bugs.gentoo.org/913070 Closes: https://bugs.gentoo.org/913103 Closes: https://bugs.gentoo.org/914159 Closes: https://bugs.gentoo.org/915054 Closes: https://bugs.gentoo.org/915119 Signed-off-by: Sam James <sam@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-3.0.52.ebuild | 235 +++++++++++++++++++++++++++++++++ 2 files changed, 236 insertions(+) |