Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 736794

Summary: sys-libs/glibc: binpkg locale-gen uses --jobs from host, and can exhaust memory on embedded systems
Product: Gentoo Linux Reporter: David Flogeras <dflogeras2>
Component: Current packagesAssignee: Gentoo Toolchain Maintainers <toolchain>
Status: RESOLVED FIXED    
Severity: normal CC: bugzilla, debug, joakim.tjernlund, jstein
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=680782
https://bugs.gentoo.org/show_bug.cgi?id=785406
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: MAKEOPTS Patch

Description David Flogeras 2020-08-11 20:31:16 UTC
It seems that when installing glibc from a binpkg, when it gets to the locale-gen stage, the following line:

locale-gen ${inplace} --jobs $(makeopts_jobs) ...

Is using makeopts_jobs from the host.  So I build packages on a machine with -j8, but this target is a Rev1 Raspberry Pi with 256MB of RAM and it cannot handle this.

Reproducible: Always
Comment 1 David Flogeras 2020-08-11 21:13:36 UTC
I think I successfully worked around it as follows:

- Use qtbz2 -s to split the package into tarball and xpak
- Use qxpak -x to unpack the xpak
- Uncompress environment.bz2, edit the offending -j8, then recompress
- qxpak -c to recreate the xpak
- qtbx2 -j to re-join the tarball and xpak into tbz2
- Then ran emaint binhost to regenerate the package metadata
Comment 2 Sergei Trofimovich (RETIRED) gentoo-dev 2020-08-11 21:15:00 UTC
Looks similar to bug #680782. Which USE-flags you are using for glibc?

USE=compile-locales is one option to avoid it.
Comment 3 David Flogeras 2020-08-11 22:26:40 UTC
Default USE for arm, which is currently:

crypt multiarch ssp static-libs -audit -caps -cet -compile-locales -custom-cflags -doc -gd -headers-only -multilib -nscd -profile -selinux -static-pie -suid -systemtap -test -vanilla


I wasn't aware of compile-locales, but wouldn't that bloat the tbz2 if it includes all locales, which I'm just going to regenerate post install?

I guess I could filter using INSTALL_MASK, but then I'd have to remember to rebuild or I have 0 locales; that's also kinda hacky.

The aforementioned bug seems to state in the last comment this was fixed in 2.29-r5.  Was this fix reverted?
Comment 4 Sergei Trofimovich (RETIRED) gentoo-dev 2020-08-11 22:42:16 UTC
I don't think anything was fixed in linked bug. The only change was to introduce USE=compile-locales.
Comment 5 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-03-25 12:51:34 UTC
*** Bug 903033 has been marked as a duplicate of this bug. ***
Comment 6 Joakim Tjernlund 2023-03-25 13:20:42 UTC
(In reply to David Flogeras from comment #3)
> Default USE for arm, which is currently:
> 
> crypt multiarch ssp static-libs -audit -caps -cet -compile-locales
> -custom-cflags -doc -gd -headers-only -multilib -nscd -profile -selinux
> -static-pie -suid -systemtap -test -vanilla
> 
> 
> I wasn't aware of compile-locales, but wouldn't that bloat the tbz2 if it
> includes all locales, which I'm just going to regenerate post install?

There was an attempt to adjust compile-locales:
  https://bugs.gentoo.org/785406
It didn't go anywhere, but you can adjust the ebuild to build just the locales you want.
Comment 7 Urban Oettli 2023-03-25 13:53:50 UTC
From my point of view it is generally wrong to take the -j value from the binary package for tasks that run locally (pkg_postinst).

Can someone explain this logic to me?

sys-libs/glibc is just one example.
More ebuilds could follow.
Comment 8 Urban Oettli 2023-03-26 17:35:13 UTC
Created attachment 859045 [details, diff]
MAKEOPTS Patch
Comment 9 Urban Oettli 2023-03-26 17:36:44 UTC
I made a quick and dirty patch which solves the problem with the MAKEOPTS (pkg_postinst).
At least in my configuration (Raspberry 3/4) it works without any problems.
But, use it at your own risk!

Probably the patch can be made much nicer and better, but I don't have the knowledge to do that.

The patch should be saved in the directory
/etc/portage/patches/sys-apps/portage/Makeopts.patch

Then remerge portage from source.
E.g. emerge -v1 portage
Comment 10 Larry the Git Cow gentoo-dev 2023-05-07 21:31:14 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=a24dfdd4b518e5d196337ffd03ce2ab8f304591d

commit a24dfdd4b518e5d196337ffd03ce2ab8f304591d
Author:     Andreas K. Hüttel <dilfridge@gentoo.org>
AuthorDate: 2023-05-07 21:30:39 +0000
Commit:     Andreas K. Hüttel <dilfridge@gentoo.org>
CommitDate: 2023-05-07 21:31:04 +0000

    sys-libs/glibc: Fix parallelization during binpkg install
    
    Bug: https://bugs.gentoo.org/736794
    Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>

 sys-libs/glibc/glibc-9999.ebuild | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
Comment 11 Andreas K. Hüttel archtester gentoo-dev 2023-05-10 19:17:21 UTC
commit 8686f981cd3d2c54826f377138ff793295c5e15d (HEAD -> master, origin/master, origin/HEAD)
Author:     Andreas K. Hüttel <dilfridge@gentoo.org>
AuthorDate: Wed May 10 21:14:22 2023 +0200
Commit:     Andreas K. Hüttel <dilfridge@gentoo.org>
CommitDate: Wed May 10 21:14:54 2023 +0200

    sys-libs/glibc: keyword 2.37-r3
    
    Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>

 sys-libs/glibc/glibc-2.37-r3.ebuild | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)