Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 785406 - sys-libs/glibc[compile-locales]: use local locales instead of installed locales
Summary: sys-libs/glibc[compile-locales]: use local locales instead of installed locales
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 2 votes (vote)
Assignee: Gentoo Toolchain Maintainers
URL:
Whiteboard:
Keywords: PATCH
Depends on:
Blocks:
 
Reported: 2021-04-24 15:27 UTC by Joakim Tjernlund
Modified: 2021-12-24 06:10 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
fix USE=compile-locales (0001-sys-libs-glibc-unbreak-USE-compile-locales.patch,1.49 KB, patch)
2021-06-23 18:47 UTC, Joakim Tjernlund
Details | Diff
v2 fix USE=compile-locales (v2-0001-sys-libs-glibc-unbreak-USE-compile-locales.patch,1.24 KB, patch)
2021-06-29 09:00 UTC, Joakim Tjernlund
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Joakim Tjernlund 2021-04-24 15:27:00 UTC
USE=compile-locales has two errors:
1) Looks for locale.gen in the build tree where there is none.
2) Looks for locale sec in the host tree, which may be old or non existent.

The patch the problem:
--- a/sys-libs/glibc/glibc-2.32-r7.ebuild
+++ b/sys-libs/glibc/glibc-2.32-r7.ebuild
@@ -1198,9 +1198,10 @@ run_locale_gen() {
        if [[ "${root}" == "--inplace-glibc" ]] ; then
                inplace="--inplace-glibc"
                root="$2"
+               export I18NPATH="${ED}"/usr/share/i18n
        fi
 
-       local locale_list="${root}/etc/locale.gen"
+       local locale_list="${EROOT}/etc/locale.gen"
 
        pushd "${ED}"/$(get_libdir) >/dev/null
Comment 1 Sergei Trofimovich (RETIRED) gentoo-dev 2021-04-24 21:46:15 UTC
I think the intent looks reasonable. San you add a few more words why I18NPATH="${ED}"/usr/share/i18n is needed?
Comment 2 Joakim Tjernlund 2021-04-24 22:02:58 UTC
Like so?
       if [[ "${root}" == "--inplace-glibc" ]] ; then
                inplace="--inplace-glibc"
                root="$2"
+               # Use locale src in build tree
+               export I18NPATH="${ED}"/usr/share/i18n
        fi
 
-       local locale_list="${root}/etc/locale.gen"
+       # Always use hosts locale config
+       local locale_list="${EROOT}/etc/locale.gen"
 
        pushd "${ED}"/$(get_libdir) >/dev/null
Comment 3 Joakim Tjernlund 2021-04-24 22:04:06 UTC
This applies to all glibc ebuild's in Gentoo tree.
Comment 4 Joakim Tjernlund 2021-05-01 11:49:55 UTC
would be really nice if this could be fixed soonish.
Comment 5 Joakim Tjernlund 2021-05-10 18:33:44 UTC
ping?
Comment 6 Joakim Tjernlund 2021-05-18 20:56:03 UTC
(In reply to Sergei Trofimovich from comment #1)
> I think the intent looks reasonable. San you add a few more words why
> I18NPATH="${ED}"/usr/share/i18n is needed?

Is something still unclear? as is now compile-locales is broken
Comment 7 Sergei Trofimovich (RETIRED) gentoo-dev 2021-05-18 23:13:34 UTC
(In reply to Joakim Tjernlund from comment #6)
> (In reply to Sergei Trofimovich from comment #1)
> > I think the intent looks reasonable. San you add a few more words why
> > I18NPATH="${ED}"/usr/share/i18n is needed?
> 
> Is something still unclear?

Yes. What does not work without it?

> as is now compile-locales is broken

What does "broken" mean? That it builds all locales, or something else on top?
Comment 8 Joakim Tjernlund 2021-05-19 07:03:27 UTC
(In reply to Sergei Trofimovich from comment #7)
> (In reply to Joakim Tjernlund from comment #6)
> > (In reply to Sergei Trofimovich from comment #1)
> > > I think the intent looks reasonable. San you add a few more words why
> > > I18NPATH="${ED}"/usr/share/i18n is needed?
> > 
> > Is something still unclear?
> 
> Yes. What does not work without it?
> 
> > as is now compile-locales is broken
> 
> What does "broken" mean? That it builds all locales, or something else on
> top?

It uses the installed locale src in ROOT which probably is old.
Comment 9 Joakim Tjernlund 2021-05-25 11:51:59 UTC
Is this still unclear? I don't understand what if so.
Comment 10 Sergei Trofimovich (RETIRED) gentoo-dev 2021-05-25 22:45:02 UTC
1. On I18NPATH=: I'm trying to wrap my head around I18NPATH="${ED}"/usr/share/i18n. I see that localedata/Makefile also sets it for localedef call. I don't understand the actual effect. Looks like without it localedef will pick /usr/share/i18n/locales definitions instead.

  Should it be better placed into `local-gen` that already has --inplace-glibc flag?

2. On 'local locale_list="${EROOT}/etc/locale.gen"' a few points:

   2a) https://dev.gentoo.org/~ulm/pms/head/pms.html says ${EROOT} is formally only available for pkg_*() functions. I believe USE=compile-locales will require it in src_install(). We will need a legal PMS workaround for it.

   2b) What path should ebould use for 'USE=compile-locales ROOT=/foo emerge -v1 glibc'? /etc/locale.gen or /foo/etc/locale.gen?
Comment 11 Joakim Tjernlund 2021-05-26 16:40:52 UTC
(In reply to Sergei Trofimovich from comment #10)
> 1. On I18NPATH=: I'm trying to wrap my head around
> I18NPATH="${ED}"/usr/share/i18n. I see that localedata/Makefile also sets it
> for localedef call. I don't understand the actual effect. Looks like without
> it localedef will pick /usr/share/i18n/locales definitions instead.

Yes, it is documented in man localedef

> 
>   Should it be better placed into `local-gen` that already has
> --inplace-glibc flag?

Not sure, but that would override a user setting I18NPATH too.

> 
> 2. On 'local locale_list="${EROOT}/etc/locale.gen"' a few points:
> 
>    2a) https://dev.gentoo.org/~ulm/pms/head/pms.html says ${EROOT} is
> formally only available for pkg_*() functions. I believe USE=compile-locales
> will require it in src_install(). We will need a legal PMS workaround for it.

yes, USE=compile-locales runs in src_install() context. Maybe just / for
compile-locales, can it be anything else when building ?

> 
>    2b) What path should ebould use for 'USE=compile-locales ROOT=/foo emerge
> -v1 glibc'? /etc/locale.gen or /foo/etc/locale.gen?

Same as 2a, / I guess.
Comment 12 Joakim Tjernlund 2021-05-26 18:53:23 UTC
(In reply to Joakim Tjernlund from comment #11)
> (In reply to Sergei Trofimovich from comment #10)

> 
> > 
> > 2. On 'local locale_list="${EROOT}/etc/locale.gen"' a few points:
> > 
> >    2a) https://dev.gentoo.org/~ulm/pms/head/pms.html says ${EROOT} is
> > formally only available for pkg_*() functions. I believe USE=compile-locales
> > will require it in src_install(). We will need a legal PMS workaround for it.
> 
> yes, USE=compile-locales runs in src_install() context. Maybe just / for
> compile-locales, can it be anything else when building ?
> 
> > 
> >    2b) What path should ebould use for 'USE=compile-locales ROOT=/foo emerge
> > -v1 glibc'? /etc/locale.gen or /foo/etc/locale.gen?
> 
> Same as 2a, / I guess.

Or one could use ESYSROOT, seems like a good fit?
Comment 13 Joakim Tjernlund 2021-06-01 08:36:48 UTC
ping ?

I think ESYSROOT is best as it mimics the non comompile-locales case.
Comment 14 Joakim Tjernlund 2021-06-04 14:09:24 UTC
Could we please sort this out ASAP? glibc is after all an important system package
Comment 15 Joakim Tjernlund 2021-06-17 21:42:04 UTC
(In reply to Joakim Tjernlund from comment #13)
> ping ?
> 
> I think ESYSROOT is best as it mimics the non comompile-locales case.

What if we just save EROOT in our own var(LOCALE_GEN_ROOT) in pkg_setup/pkg_pretend
and use that? Something like this:

--- glibc-2.33.ebuild	2021-06-13 18:54:12.655146226 +0200
+++ glibc-2.33.ebuild	2021-06-17 23:40:27.704909576 +0200
@@ -719,6 +719,7 @@
 	# All the checks...
 	einfo "Checking general environment sanity."
 	sanity_prechecks
+	LOCALE_GEN_ROOT="${EROOT}"
 }
 
 pkg_setup() {
@@ -1175,9 +1176,10 @@
 	if [[ "${root}" == "--inplace-glibc" ]] ; then
 		inplace="--inplace-glibc"
 		root="$2"
+		export I18NPATH="${ED}"/usr/share/i18n
 	fi
 
-	local locale_list="${root}/etc/locale.gen"
+	local locale_list="${LOCALE_GEN_ROOT}/etc/locale.gen"
 
 	pushd "${ED}"/$(get_libdir) >/dev/null
Comment 16 Joakim Tjernlund 2021-06-23 12:14:43 UTC
Ping? 
w.r.t 2b) I think that is a matter for another bug, this fix will unbreak compile-locales so it works the same way as !compile-locales

I really need to get this fixed so pretty please ... I can attach a proper patch
if you prefer that?
Comment 17 Sergei Trofimovich (RETIRED) gentoo-dev 2021-06-23 17:11:06 UTC
Proper tested patch will help. Use of ${ESYSROOT} sounds reasonable.
Comment 18 Joakim Tjernlund 2021-06-23 18:20:03 UTC
(In reply to Sergei Trofimovich from comment #17)
> Proper tested patch will help. Use of ${ESYSROOT} sounds reasonable.

I do a proper patch, but why ESYSROOT ? I only suggested that because EROOT
was not allowed to use inside src_install. Now I got a fix for that.
Comment 19 Joakim Tjernlund 2021-06-23 18:47:59 UTC
Created attachment 718281 [details, diff]
fix USE=compile-locales

Fixes the two problems mentioned in this bug.
Comment 20 Sergei Trofimovich (RETIRED) gentoo-dev 2021-06-24 17:25:47 UTC
(In reply to Joakim Tjernlund from comment #19)
> Created attachment 718281 [details, diff] [details, diff]
> fix USE=compile-locales
> 
> Fixes the two problems mentioned in this bug.

> --- a/sys-libs/glibc/glibc-2.33.ebuild
> +++ b/sys-libs/glibc/glibc-2.33.ebuild
> @@ -724,6 +724,7 @@ pkg_pretend() {
>  pkg_setup() {
>  	# see bug 682570
>  	[[ -z ${BOOTSTRAP_RAP} ]] && python-any-r1_pkg_setup
> +	LOCALE_GEN_ROOT="${EROOT}" # For locale-gen later
>  }

We should not use ${EROOT} indirectly in src_compile() phase just by storing the variable in pkg_*() phase. Using ${ESYSROOT} should be fine.
 
>  # src_unpack
> @@ -1175,9 +1176,12 @@ run_locale_gen() {
>  	if [[ "${root}" == "--inplace-glibc" ]] ; then
>  		inplace="--inplace-glibc"
>  		root="$2"
> +		# Use locale src in build tree
> +		export I18NPATH="${ED}"/usr/share/i18n

`I18NPATH` belongs to locale-gen script. Also manpage for `localdef` says current directory is inspected. Do we run in a wrong directory?

>  	fi
>  
> -	local locale_list="${root}/etc/locale.gen"
> +	# Always use hosts(EROOT) locale config
> +	local locale_list="${LOCALE_GEN_ROOT}/etc/locale.gen"

I think it should just use 

    local locale_list="${ESYSROOT}/etc/locale.gen"

>  	pushd "${ED}"/$(get_libdir) >/dev/null
Comment 21 Joakim Tjernlund 2021-06-29 08:37:19 UTC
Never got this comment in my mailbox, strange ...

(In reply to Sergei Trofimovich from comment #20)
> (In reply to Joakim Tjernlund from comment #19)
> > Created attachment 718281 [details, diff] [details, diff] [details, diff]
> > fix USE=compile-locales
> > 
> > Fixes the two problems mentioned in this bug.
> 
> > --- a/sys-libs/glibc/glibc-2.33.ebuild
> > +++ b/sys-libs/glibc/glibc-2.33.ebuild
> > @@ -724,6 +724,7 @@ pkg_pretend() {
> >  pkg_setup() {
> >  	# see bug 682570
> >  	[[ -z ${BOOTSTRAP_RAP} ]] && python-any-r1_pkg_setup
> > +	LOCALE_GEN_ROOT="${EROOT}" # For locale-gen later
> >  }
> 
> We should not use ${EROOT} indirectly in src_compile() phase just by storing
> the variable in pkg_*() phase. Using ${ESYSROOT} should be fine.

OK, I will adjust to ${ESYSROOT} then

>  
> >  # src_unpack
> > @@ -1175,9 +1176,12 @@ run_locale_gen() {
> >  	if [[ "${root}" == "--inplace-glibc" ]] ; then
> >  		inplace="--inplace-glibc"
> >  		root="$2"
> > +		# Use locale src in build tree
> > +		export I18NPATH="${ED}"/usr/share/i18n
> 
> `I18NPATH` belongs to locale-gen script. Also manpage for `localdef` says
> current directory is inspected. Do we run in a wrong directory?

Sort of, ebuild does pushd "${ED}"/$(get_libdir) >/dev/null to get
at glibc libs. Specifying I18NPATH makes sure we use correct locale data
regardless of where localedef is executed.

> 
> >  	fi
> >  
> > -	local locale_list="${root}/etc/locale.gen"
> > +	# Always use hosts(EROOT) locale config
> > +	local locale_list="${LOCALE_GEN_ROOT}/etc/locale.gen"
> 
> I think it should just use 
> 
>     local locale_list="${ESYSROOT}/etc/locale.gen"

Will do.

> 
> >  	pushd "${ED}"/$(get_libdir) >/dev/null
Comment 22 Joakim Tjernlund 2021-06-29 09:00:12 UTC
Created attachment 720090 [details, diff]
v2 fix USE=compile-locales

built and tested:
ocale-gen --inplace-glibc --jobs 8 --config /etc/locale.gen --destdir /var/tmp/portage/sys-libs/glibc-2.33/image/
 * Building locales in DESTDIR '/var/tmp/portage/sys-libs/glibc-2.33/image/'
 * Generating 5 locales (this might take a while) with 8 jobs
 *  (5/5) Generating C.UTF-8 ...                                                                      [ ok ]
 *  (3/5) Generating en_US.UTF-8 ...                                                                  [ ok ]
 *  (2/5) Generating en_GB.UTF-8 ...                                                                  [ ok ]
 *  (1/5) Generating en_DK.UTF-8 ...                                                                  [ ok ]
 *  (4/5) Generating sv_SE.UTF-8 ...                                                                  [ ok ]
 * Generation complete
 * Adding locales to archive ...
Comment 23 Joakim Tjernlund 2021-06-30 17:20:51 UTC
Happy now?
Comment 24 Andreas K. Hüttel archtester gentoo-dev 2021-07-02 20:56:24 UTC
OK this went around in a few cycles, but there's a misunderstanding here:

compile-locales is intended to build *all* locales. If it doesn't do that then it's a bug...
Comment 25 Joakim Tjernlund 2021-07-03 10:37:33 UTC
(In reply to Andreas K. Hüttel from comment #24)
> OK this went around in a few cycles, but there's a misunderstanding here:
> 
> compile-locales is intended to build *all* locales. If it doesn't do that
> then it's a bug...

what? This is not how the code is structured/intended. There is a clear
intention to build what is in locale.gen.
Comment 26 Sergei Trofimovich (RETIRED) gentoo-dev 2021-07-05 22:38:32 UTC
I think it's reasonable to provide a knob for USE=compile-locales to use only a subset of locales when needed.

But it's a good point. There should be something distinguishing the intent to build all locales and to build defined set of them.
Comment 27 Joakim Tjernlund 2021-07-10 19:07:09 UTC
(In reply to Sergei Trofimovich from comment #26)
> I think it's reasonable to provide a knob for USE=compile-locales to use
> only a subset of locales when needed.
> 
> But it's a good point. There should be something distinguishing the intent
> to build all locales and to build defined set of them.

(late reply, been away this week)

I think this issue is snowballing into much more than it intended to fix.
Can we just correct the problem at hand and discuss further tweaks in a separate bug?
Comment 28 Joakim Tjernlund 2021-07-15 13:30:45 UTC
ping?
Comment 29 Joakim Tjernlund 2021-08-05 21:38:22 UTC
ping?
Comment 30 Matt Turner gentoo-dev 2021-08-06 23:42:09 UTC
(In reply to Joakim Tjernlund from comment #25)
> (In reply to Andreas K. Hüttel from comment #24)
> > OK this went around in a few cycles, but there's a misunderstanding here:
> > 
> > compile-locales is intended to build *all* locales. If it doesn't do that
> > then it's a bug...
> 
> what? This is not how the code is structured/intended. There is a clear
> intention to build what is in locale.gen.

Other than savedconfig, I can't think of cases where a file outside of the portage configuration influences what's installed on disk like this.

I think you might be a little presumptuous in claiming that there's clear intention one way or another. Andreas is the one that added USE=compile-locales support (at my request, for RelEng builds).

I'm not really involved here, but if this package were my responsibility, I would be disinclined to put a lot of effort into this bug because I find the pings and questions like "Could we please sort this out ASAP?" very off-putting.

I'd suggest writing and attaching a git formatted patch with a well written explanation in the commit message of: a) what the problem is, b) how the patch resolves it. As it is, participants in this bug are spending more time just trying to understand you than anything else.
Comment 31 Joakim Tjernlund 2021-11-02 22:21:28 UTC
(In reply to Matt Turner from comment #30)
> (In reply to Joakim Tjernlund from comment #25)
> > (In reply to Andreas K. Hüttel from comment #24)
> > > OK this went around in a few cycles, but there's a misunderstanding here:
> > > 
> > > compile-locales is intended to build *all* locales. If it doesn't do that
> > > then it's a bug...
> > 
> > what? This is not how the code is structured/intended. There is a clear
> > intention to build what is in locale.gen.
> 
> Other than savedconfig, I can't think of cases where a file outside of the
> portage configuration influences what's installed on disk like this.
> 
> I think you might be a little presumptuous in claiming that there's clear
> intention one way or another. Andreas is the one that added

I think code like this clearly spells intention: 
   locale_list="${root}/etc/locale.gen

> USE=compile-locales support (at my request, for RelEng builds).

Then you should have some interest in getting this sorted, right?

> 
> I'm not really involved here, but if this package were my responsibility, I
> would be disinclined to put a lot of effort into this bug because I find the
> pings and questions like "Could we please sort this out ASAP?" very
> off-putting.

Well, agreed that such things can be a bit annoying but at least there was
almost a month between the last pings.

> 
> I'd suggest writing and attaching a git formatted patch with a well written
> explanation in the commit message of: a) what the problem is, b) how the
> patch resolves it. As it is, participants in this bug are spending more time
> just trying to understand you than anything else.

I think I have already done that, before you commented. Please read the
bug/patch in here.

Anyhow, consider this my last "ping", cheers.