Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 680782 - =sys-libs/glibc-2.28-r5 - pkg_postinst(): run_locale_gen() uses MAKEOPTS from build target
Summary: =sys-libs/glibc-2.28-r5 - pkg_postinst(): run_locale_gen() uses MAKEOPTS fro...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Toolchain Maintainers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-03-17 15:05 UTC by tt_1
Modified: 2019-09-18 15:28 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
patch against glibc-2.29-r2 ebuild (glibc-2.29.patch,1.09 KB, patch)
2019-08-11 10:54 UTC, tt_1
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description tt_1 2019-03-17 15:05:13 UTC
yesterday an update for glibc to 2.28 came in for arm, and so I decided to cross-compile it from an amd64 host with 12 cores available. 

When this binary was emerged on the host with the -k switch of emerge, it ran locale-gen with -j12 as well. In my case it didn't matter much, I have my locales restricted to four anyway and the device has four cores available. 

Nontheless, this shouldn't happen, as it can possibly cause destruction if the discrepance of cores between host and target is way higher, and the amount of locales to be generated much higher.
Comment 1 Sergei Trofimovich gentoo-dev 2019-03-17 16:56:17 UTC
locale-gen always runs with '--jobs $(makeopts_jobs)' parameter:
    https://gitweb.gentoo.org/repo/gentoo.git/tree/sys-libs/glibc/glibc-2.28-r5.ebuild#n1113

https://dev.gentoo.org/~ulm/pms/head/pms.html does not say anything special about MAKEOPTS. It means MAKEOPTS is stored in environment as any other variable and is not dynamically changeable in pkg_*() phases.

$ cat b680782-0.ebuild 
    EAPI=6
    inherit multiprocessing
    DESCRIPTION="Reproducer for https://bugs.gentoo.org/680782"
    HOMEPAGE="https://bugs.gentoo.org/680782"
    KEYWORDS="amd64 x86"
    LICENSE="GPL-2"
    SLOT="0"
    S="${WORKDIR}"
    src_install() { einfo "src_install() jobs: $(makeopts_jobs)"; }
    pkg_postinst() { einfo "src_install() jobs: $(makeopts_jobs)"; }

# MAKEOPTS=-j4 emerge --buildpkgonly app-misc/b680782 --quiet-build=n
 * src_install() jobs: 4
# MAKEOPTS=-j2 emerge --usepkg app-misc/b680782 --quiet-build=n
 * src_install() jobs: 4

We could do something for ${MERGE_TYPE} == "binary" 

> Nontheless, this shouldn't happen, as it can possibly cause destruction if the discrepance of cores between host and target is way higher, and the amount of locales to be generated much higher.

Running 400 processes is totally fine on a single core as long as RAM used for locale-generation is reasonable.
Comment 2 tt_1 2019-03-17 17:02:56 UTC
restraining to -j1 for ${MERGE_TYPE} == "binary" would be just fine in my opinion.
Comment 3 Sergei Trofimovich gentoo-dev 2019-03-17 17:30:13 UTC
(In reply to tt_1 from comment #2)
> restraining to -j1 for ${MERGE_TYPE} == "binary" would be just fine in my
> opinion.

It would slow glibc installation down for everyone who builds binary packages locally just before an upgrade.
Comment 4 tt_1 2019-03-17 18:06:10 UTC
Do you mean in case of 

FEATURES="buildpkg" 

?
Comment 5 Alexander Tsoy 2019-04-10 23:08:44 UTC
Got the same problem. 6 localedef processes is too much for my arm board:

$ sudo dmesg | grep -i killed
[423527.159682] Killed process 937 (localedef) total-vm:77484kB, anon-rss:74584kB, file-rss:24kB, shmem-rss:0kB

I thinks it's clearly a package manager issue. It should restore MAKEOPTS to its original state after restoring environment at least in pkg_* phases... or doesn't store MAKEOPTS in environment in the first place.
Comment 6 Sergei Trofimovich gentoo-dev 2019-04-11 09:41:55 UTC
(In reply to Alexander Tsoy from comment #5)
> Got the same problem. 6 localedef processes is too much for my arm board:
> 
> $ sudo dmesg | grep -i killed
> [423527.159682] Killed process 937 (localedef) total-vm:77484kB,
> anon-rss:74584kB, file-rss:24kB, shmem-rss:0kB
> 
> I thinks it's clearly a package manager issue. It should restore MAKEOPTS to
> its original state after restoring environment at least in pkg_* phases...
> or doesn't store MAKEOPTS in environment in the first place.

As a workaround you can set lower MAKEOPTS on a host machine where you build glibc.

Gentoo's package manager specification does not say anything special about MAKEOPTS. Hence it will be preserved in the environmet as the rest of options.

Thus I would say it's not a problem in package manager but in a glibc ebuild.
Comment 7 tt_1 2019-07-22 11:59:15 UTC
So, what is the status of this? Do you deem it to be fixable by dropping to -j1 with some fancy guards for merge type binary, in src_install? If so, I might be able to cook up something and attach a possible patch for the glibc ebuild. 

Thanks
Comment 8 Sergei Trofimovich gentoo-dev 2019-07-22 19:00:08 UTC
I'm not aware of any work done in this area thus status is unchanged: we need a tweak in glibc ebuild.

I don't think there is a way to implement it cleanly. Aside from moving locale generation into another package.

'[[ ${MERGE_TYPE} = "binary" ]]' might might be goot enough compromise. It will slow down all binary package users though.
Comment 9 tt_1 2019-08-11 10:54:55 UTC
Created attachment 586536 [details, diff]
patch against glibc-2.29-r2 ebuild

Would this attempt work? In case you were willing to slow down locale-gen to -j1 for everyone who uses a binpkg, no matter how strong the hardware? :-)
Comment 10 Sergei Trofimovich gentoo-dev 2019-08-11 16:40:04 UTC
(In reply to tt_1 from comment #9)
> Created attachment 586536 [details, diff] [details, diff]
> patch against glibc-2.29-r2 ebuild
> 
> Would this attempt work? In case you were willing to slow down locale-gen to
> -j1 for everyone who uses a binpkg, no matter how strong the hardware? :-)

> if [[ ${MERGE_TYPE} == !"binary" ]] ; then

That is not a valid match syntax.

You might want to look at USE=compile-locales added no recent glibcs. That shifts locale generation from pkg_*() to src_*() phases and makes locale-gen a no-op.
Comment 11 tt_1 2019-09-18 15:28:17 UTC
fixed in glibc-2.29-r5! :)