Bug 697908 – sys-libs/glibc-2.29-r2 - /usr/sbin/locale-gen returns "success" when it fails to generate locales based on a corrupt /etc/locale.gen

• Product: Gentoo Linux
• Status: RESOLVED FIXED

[Description Chris Slycord 2019-10-17 06:30:58 UTC]

My /etc/locale.gen file was corrupted with windows line endings (user error; copied over from an editor made on a windows box).

Upon running locale-gen, the output gave errors of "failed to set locale" and others related to unknown characters in a locale file. locale-gen failed because it treated the unprintable characters as being part of the locale name.

The bug isn't that it failed to generate locales; that's my fault.

The bug is that it gives a "successful" exit code after failing to generate locales and displaying errors.

Reproducible: Always

Steps to Reproduce:
1. create locale.gen file with invalid line endings
2. run "locale-gen"
3. run "$?"
Actual Results:  
locale-gen generates errors, fails to build any locales, but gives a "success" exit code ($? output is 0)

Expected Results:  
Give a non-zero/unsuccessful exit code upon failing to build locales.

[Comment 1 Jeroen Roovers (RETIRED) 2019-10-17 06:34:19 UTC]

Please post your `emerge --info' output in a comment.

[Comment 2 Chris Slycord 2019-10-17 08:30:26 UTC]

Created attachment 592972 [details]
emerge --info

[Comment 3 Chris Slycord 2019-10-17 08:35:40 UTC]

Created attachment 592974 [details]
locale-gen output along with success from $?

[Comment 4 Chris Slycord 2019-10-17 08:38:47 UTC]

Created attachment 592976 [details]
locale-gen output redirected to file

If it helps, this shows that it's trying to use the unprintable "^M" character at the end of each name/line from /etc/locale.gen, since they were erroneously placed in the file.

[Comment 5 Sergei Trofimovich (RETIRED) 2019-10-20 07:54:29 UTC]

Can you also attach broken locale.gen?

[Comment 6 Chris Slycord 2019-10-24 11:57:04 UTC]

Created attachment 593758 [details]
locale.gen (with windows line endings)

locale-gen failed with this file. But returned success.

[Comment 7 Sergei Trofimovich (RETIRED) 2019-11-15 21:41:48 UTC]

Confirmed locally. The output is in the form:

sf / # locale-gen
 * Generating 5 locales (this might take a while) with 8 jobs
 ...(3/5) Generating ko_KR.EUC-KR
failed to set locale!
' not found: No such file or directory
failed to set locale!
/usr/share/i18n/locales/ko_KR:6141: LC_TIME: unknown character in field `abday'
failed to set locale!
...
[error] LC_MESSAGES: value for field `noexpr' must not be an empty string                                                      [ ok ]
 ...(2/5) Generating en_US.UTF-8
failed to set locale!
' not found: No such file or directory                                                                                         [ ok ]
 * Generation complete
 * Adding locales to archive ...                                                                                               [ ok ]
sf / # locate locale-gen

[Comment 8 Sergei Trofimovich (RETIRED) 2019-11-15 21:58:32 UTC]

Looks like it's at least a deficiency in localedef's exit status:

# /usr/bin/localedef -c --no-archive -i en_US -f $'ISO-8859-1\r' -A /usr/share/locale/locale.alias --prefix / en_US
failed to set locale!
' not found: No such file or directory

# echo $?
0

Fun fact: $'\r' actually masks full error:

# /usr/bin/localedef -c --no-archive -i en_US -f $'ISO-8859-1\r' -A /usr/share/locale/locale.alias --prefix / en_US 2>&1 | cat -v
failed to set locale!
[error] character map file `ISO-8859-1^M' not found: No such file or directory

[Comment 9 Sergei Trofimovich (RETIRED) 2019-11-15 22:10:39 UTC]

'man localedef' says:

"""
    EXIT STATUS
       One of the following exit values can be returned by localedef:

          0         Command completed successfully.

          1         Warnings or errors occurred, output files were written.

          4         Errors encountered, no output created.
"""

I think we should have received at least '1' if not '4'. I declare it a localedef bug.

[Comment 10 Sergei Trofimovich (RETIRED) 2019-11-15 22:18:01 UTC]

Filed upstream bug as https://sourceware.org/PR25195

[Comment 11 Sergei Trofimovich (RETIRED) 2019-11-21 07:29:00 UTC]

In upstream bug Carlos discovered we are using '-c' option. Let's try to get rid of that.

[Comment 12 Larry the Git Cow 2019-11-21 07:54:02 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1968d85e42382fa4820f6978e116abe091ac4825

commit 1968d85e42382fa4820f6978e116abe091ac4825
Author:     Sergei Trofimovich <[e-mail]>
AuthorDate: 2019-11-21 07:53:41 +0000
Commit:     Sergei Trofimovich <[e-mail]>
CommitDate: 2019-11-21 07:53:56 +0000

    sys-libs/glibc: drop -c (--force) flag from localedef
    
    As found out in https://sourceware.org/PR25195
    -c/--force makes 'localedef' to ignore all errors.
    
    The change drops -c flag and relies on clean run.
    
    Reported-by: Chris Slycord
    Bug: https://bugs.gentoo.org/697908
    Package-Manager: Portage-2.3.79, Repoman-2.3.18
    Signed-off-by: Sergei Trofimovich <[e-mail]>

 sys-libs/glibc/Manifest          | 2 +-
 sys-libs/glibc/glibc-9999.ebuild | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

[Comment 13 Sergei Trofimovich (RETIRED) 2019-11-22 20:20:12 UTC]

Let's close it as fixed and get that fix into glibc-2.31. As it was broken forever I suggest not to spent much time to port it back.