Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 912054 - dev-python/numexpr-2.8.4: sandbox causes memory allocation to fail during emerge with ample memory in reserve
Summary: dev-python/numexpr-2.8.4: sandbox causes memory allocation to fail during eme...
Status: RESOLVED NEEDINFO
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Sandbox (show other bugs)
Hardware: AMD64 Linux
: Normal normal (vote)
Assignee: Sandbox Maintainers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-08-11 04:41 UTC by 0x5a80
Modified: 2023-10-23 06:23 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info (emerge_info.txt,6.07 KB, text/plain)
2023-08-11 04:41 UTC, 0x5a80
Details
build log (numexpr-build.txt,4.26 KB, text/plain)
2023-08-11 04:42 UTC, 0x5a80
Details
xz-compressed strace output including clone call failing to allocate memory (strace_out.txt.xz,55.76 KB, application/x-xz)
2023-08-11 04:42 UTC, 0x5a80
Details

Note You need to log in before you can comment on or make changes to this bug.
Description 0x5a80 2023-08-11 04:41:36 UTC
Created attachment 867599 [details]
emerge --info

When emerging dev-python/numexpr, the emerge fails with: "error: command '/usr/bin/x86_64-pc-linux-gnu-gcc' failed: Cannot allocate memory". This error does not occur if emerge is invoked with `FEATURES=-usersandbox`.

To reproduce the problem outside of emerge, cd to the workdir (/var/tmp/portage/dev-python/numexpr-2.8.4/work/numexpr-2.8.4 in my case) and then run `sandbox python3.11 setup.py build_ext -j 1`. 

strace indicates the failed allocation is in the following syscall:
[pid  9362] clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f33d3d90990) = -1 ENOMEM (Cannot allocate memory)

I have attached my emerge --info and am adding supplementary attachments with build log and compressed strace log.
Comment 1 0x5a80 2023-08-11 04:42:05 UTC
Created attachment 867600 [details]
build log
Comment 2 0x5a80 2023-08-11 04:42:55 UTC
Created attachment 867601 [details]
xz-compressed strace output including clone call failing to allocate memory
Comment 3 Mike Gilbert gentoo-dev 2023-08-11 19:40:56 UTC
I am unable to reproduce the issue.
Comment 4 0x5a80 2023-08-11 20:51:35 UTC
What other information can I provide to help in reproducing the problem?
Comment 5 Mike Gilbert gentoo-dev 2023-08-12 02:14:17 UTC
If you can provide steps to reproduce the issue from a fresh stage3, that would help. At the moment, I have no idea what unique aspect of your system triggers the issue.
Comment 6 Mike Gilbert gentoo-dev 2023-08-12 19:51:17 UTC
My best guess is that the kernel is enforcing some limit on memory, and running the build within the sandbox is just enough to exceed that limit.

I would look for any suspiciously low limits in /etc/security/limits.conf.

I also wonder if you have customized these sysctl settings:

vm.overcommit_memory
vm.overcommit_ratio

These affect the amount of memory the kernel will permit a cloned process to use.

The configuration of numpy and its dependencies might also have some effect. Locally, I usually have the "lapack" USE flag on numpy disabled to minimize the dependency footprint. I tried enabling "lapack" as a test, but that had no effect.

numpy[lapack] also has some dependencies that optionally depend code written in fortran. I would prefer not to rebuild them with fortran enabled unless there is some evidence that triggers the issue.
Comment 7 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-08-12 19:55:40 UTC
(In reply to Mike Gilbert from comment #6)
> numpy[lapack] also has some dependencies that optionally depend code written
> in fortran. I would prefer not to rebuild them with fortran enabled unless
> there is some evidence that triggers the issue.

To this end, a list of other packages which fail could be helpful.