It is not possible to emerge certain packages with certain locale settings, for example: app-text/gnome-doc-utils-0.20.10-r3 dev-ruby/rdoc-6.5.0 x11-libs/libxcb-1.16.1 (there are several more packages like that, but I don't currently remember which) fail to emerge on my system unless I set 'LC_ALL=' first. They produce errors like: UnicodeEncodeError: 'latin-1' codec can't encode characters in position 3583-3588: ordinal not in range(256) UnicodeEncodeError: 'latin-1' codec can't encode character '\u201c' in position 79: ordinal not in range(256) /usr/lib64/ruby/gems/3.1.0/gems/kpeg-1.3.3/lib/kpeg/grammar.rb:123:in `initialize': invalid character property name {Zl}: /\\n|\\r\\n?|\\p{Zl}|\\p{Zp}/ (RegexpError) My 'LC_ALL' is 'en_US' - I expect this to be rather standard. Reproducible: Always Actual Results: I expect emerge shouldn't fail/succeed depending on the current locale. Expected Results: I expect Portage to be compatible with all the locale settings.
Created attachment 889362 [details] 'emerge --info' output
Comment on attachment 889362 [details] 'emerge --info' output >Portage 3.0.61 (python 3.11.8-final-0, default/linux/amd64/23.0/desktop/plasma/systemd, gcc-13, glibc-2.38-r10, 6.6.13-gentoo x86_64) >================================================================= >(...) > >sh bash 5.1_p16-r6 >ld GNU ld (Gentoo 2.41 p5) 2.41.0 >app-misc/pax-utils: 1.3.7::gentoo >app-shells/bash: 5.1_p16-r6::gentoo >dev-build/autoconf: 2.13-r8::gentoo, 2.71-r6::gentoo >dev-build/automake: 1.16.5-r2::gentoo >dev-build/cmake: 3.28.3::gentoo >dev-build/libtool: 2.4.7-r4::gentoo >dev-build/make: 4.4.1-r1::gentoo >dev-build/meson: 1.3.2::gentoo >dev-java/java-config: 2.3.3-r1::gentoo >dev-lang/perl: 5.38.2-r2::gentoo >dev-lang/python: 3.11.8_p1::gentoo, 3.12.2_p1::gentoo >dev-lang/rust: 1.75.0-r1::gentoo >sys-apps/baselayout: 2.14-r2::gentoo >sys-apps/sandbox: 2.38::gentoo >sys-apps/systemd: 255.3-r1::gentoo >sys-devel/binutils: 2.41-r5::gentoo >sys-devel/binutils-config: 5.5::gentoo >sys-devel/clang: 15.0.7-r3::gentoo, 16.0.6::gentoo, 17.0.6::gentoo >sys-devel/gcc: 13.2.1_p20240210::gentoo >sys-devel/gcc-config: 2.11::gentoo >sys-devel/lld: 15.0.7::gentoo, 17.0.6::gentoo >sys-devel/llvm: 15.0.7-r3::gentoo, 16.0.6::gentoo, 17.0.6::gentoo >sys-kernel/linux-headers: 6.6-r1::gentoo (virtual/os-headers) >sys-libs/glibc: 2.38-r10::gentoo >Repositories: > >gentoo > location: /var/db/repos/gentoo > sync-type: rsync > sync-uri: rsync://rsync.gentoo.org/gentoo-portage > priority: -1000 > volatile: False > sync-rsync-verify-jobs: 1 > sync-rsync-extra-opts: > sync-rsync-verify-metamanifest: yes > sync-rsync-verify-max-age: 24 > >localrepo > location: /var/db/repos/localrepo > masters: gentoo > volatile: False > >steam-overlay > location: /var/db/repos/steam-overlay > sync-type: git > sync-uri: https://github.com/gentoo-mirror/steam-overlay.git > masters: gentoo > volatile: False > >ACCEPT_KEYWORDS="amd64" >ACCEPT_LICENSE="@FREE @FREE @BINARY-REDISTRIBUTABLE CC-BY-ND-3.0 free-noncomm BUILDLIC DOOM-COLLECTORS-EDITION GOG_EULA HRP MPEG-4 RAR XMAME Microsoft-vscode Activision ChexQuest3 Mojang RtMidi Sublime ValveSteamLicense WidePix android dropbox google-chrome hplip-plugin lha truecrypt-3.0 AMD-GPU-PRO-EULA PUEL-11" >CBUILD="x86_64-pc-linux-gnu" >CFLAGS="-march=x86-64-v3 -mtune=native -O2 -pipe" >CHOST="x86_64-pc-linux-gnu" >CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt" >CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d" >CXXFLAGS="-march=x86-64-v3 -mtune=native -O2 -pipe" >DISTDIR="/var/cache/distfiles" >EMERGE_DEFAULT_OPTS="--keep-going=y" >ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME" >FCFLAGS="-march=x86-64-v3 -mtune=native -O2 -pipe" >FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" >FFLAGS="-march=x86-64-v3 -mtune=native -O2 -pipe" >GENTOO_MIRRORS="rsync://ftp.vectranet.pl/gentoo/" >LANG="C" >LC_ALL="en_US" >LDFLAGS="-fuse-ld=mold -Wl,-O1 -Wl,--as-needed" >LEX="flex" >LINGUAS="de en fr pl" >MAKEOPTS="-j32" >PKGDIR="/var/cache/binpkgs" >PORTAGE_CONFIGROOT="/" >PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" >PORTAGE_TMPDIR="/var/tmp" >SHELL="/bin/bash" >USE="X a52 aac acl acpi activities alsa amd64 bluetooth bluray bmp branding bs2b bzip2 cairo cdda cdr cet clamav codec2 crypt cups dav1d dbus declarative dri dts dvd dvdr encode exif faad flac gdbm gif gpm gsm gtk gui iconv icu ipv6 jpeg jpeg2k jpg kde kwallet lcms libnotify libsamplerate libtirpc lz4 lzma mad matroska mng modplug mp3 mp4 mpeg multilib musepack ncurses networkmanager nls nvenc ogg opencl openexr opengl openh264 openmp opus pam pango pcre pcx pdf pipewire plasma png policykit ppds pulseaudio qml qt5 readline screencast sdl seccomp semantic-desktop sound speex spell ssl startup-notification svg systemd taglib test-rust tga theora tiff truetype udev udisks unicode upower usb vaapi vcd vdpau vorbis vpx vulkan wavpack wayland webp widgets wmf wxwidgets x264 x265 xattr xcb xft xml xpm xv xvid zlib zstd" ABI_X86="64" ADA_TARGET="gcc_12" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words stage" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" INPUT_DEVICES="libinput joystick roccat_koneaimo" KERNEL="linux" L10N="de en en-US fr pl" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" QEMU_SOFTMMU_TARGETS="x86_64 m68k ppc" RUBY_TARGETS="ruby31" VIDEO_CARDS="amdgpu intel radeon radeonsi" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account" >Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LD, LFLAGS, LIBTOOL, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
Please use the LANG variable, not LC_ALL.
The only way Portage or ebuilds could "fix" this would be to ignore your LC_ALL setting. The normal way to set the locale on your system is via the LANG variable. Portage and ebuilds are able to override aspects of LANG by setting specific LC variables. LC_ALL overrides LANG and all of the other LC variables and should rarely be used in practice.
I don't know much about the Portage internals, but compilation succeeding or failing depending on the hist system locale settings looks very very wrong to me. Removing LC_ALL and setting LANG fixes the compilation problem, but creates a new issue for me - each time I execute emerge command, I see the following message (twice) in the terminal emulator: setlocale: unsupported locale setting setlocale: unsupported locale setting What's interesting, this only happens within terminal emulator, inside KDE - if I log in on the console (not to X11/KDE), the glitch is not visible. Here are the language-related variables from my KDE session (variables are set by KDE): LANGUAGE=C LANG=C LC_ADDRESS=C LC_NAME=C LC_MONETARY=pl_PL.UTF-8 LC_PAPER=C LC_TELEPHONE=pl_PL.US-ASCII LC_MEASUREMENT=C LC_TIME=C LC_COLLATE=szl_PL.UTF-8 LC_NUMERIC=C and here is what I have outside of KDE (this works): LANG=C.UTF-8 It would be nice if Portage could at least write out what is the problem with locale - the enigmatic 'setlocale: unsupported locale setting' tells very little about what is actually wrong.
(In reply to Roman from comment #5) > Removing LC_ALL and setting LANG fixes the compilation problem, but creates > a new issue for me - each time I execute emerge command, I see the following > message (twice) in the terminal emulator: > > setlocale: unsupported locale setting > setlocale: unsupported locale setting > > It would be nice if Portage could at least write out what is the problem > with locale - the enigmatic 'setlocale: unsupported locale setting' tells > very little about what is actually wrong. I looked up the code that emits this message, and the failing function call is locale.setlocale(locale.LC_ALL, '') as suggested in the docs here: https://docs.python.org/3/library/locale.html#locale.setlocale Our code oddly calls setlocale many times in a loop since this commit: From ccb1377a18ce8addb21000b0009e360c2641aa8c Mon Sep 17 00:00:00 2001 From: Arfrever Frehtes Taifersar Arahesis <Arfrever@Apache.Org> Date: Fri, 13 Nov 2015 22:50:22 +0100 Subject: [PATCH] _emerge.main.emerge_main(): Make locale variables from configuration files affect locale of emerge process. > # Make locale variables from configuration files (make.defaults, make.conf) affect locale of emerge process. > for locale_var_name in ( > "LANGUAGE", > "LC_ALL", > "LC_ADDRESS", > "LC_COLLATE", > "LC_CTYPE", > "LC_IDENTIFICATION", > "LC_MEASUREMENT", > "LC_MESSAGES", > "LC_MONETARY", > "LC_NAME", > "LC_NUMERIC", > "LC_PAPER", > "LC_TELEPHONE", > "LC_TIME", > "LANG", > ): > locale_var_value = emerge_config.running_config.settings.get(locale_var_name) > if locale_var_value is not None: > os.environ.setdefault(locale_var_name, locale_var_value) > try: > locale.setlocale(locale.LC_ALL, "") > except locale.Error as e: > writemsg_level(f"setlocale: {e}\n", level=logging.WARN)
When setlocale fails, I suppose we can output a diff of the locale variables compared to a previous successful setlocale call if available, and otherwise just output all the locale variables.
(In reply to Roman from comment #5) > What's interesting, this only happens within terminal emulator, inside KDE - > if I log in on the console (not to X11/KDE), the glitch is not visible. Here > are the language-related variables from my KDE session (variables are set by > KDE): > > LANGUAGE=C > LANG=C > LC_ADDRESS=C > LC_NAME=C > LC_MONETARY=pl_PL.UTF-8 > LC_PAPER=C > LC_TELEPHONE=pl_PL.US-ASCII > LC_MEASUREMENT=C > LC_TIME=C > LC_COLLATE=szl_PL.UTF-8 > LC_NUMERIC=C You can probably straighten this out by reverting to defaults for Regional Settings in the System Settings app.
Well, I would prefer not to lose the ability to set the locale to something other than default.
Using a non-default locale is generally fine. Just do it via LANG and/or the LC variables. Setting LC_ALL is what breaks packages, since they can't override it when necessary without ignoring your LC_ALL setting. You are welcome to file individual bugs against packages that break with LC_ALL set, but be prepared to have them closed as WONTFIX.
In particular, having LC_ALL set to a non-UTF-8 locale is known to cause breakage.
