Bug 915894 – sys-apps/openrc-0.51 breaks cgroupsv2 for e.g. dev-java/openjdk

• Status: RESOLVED FIXED
• Product: Gentoo Hosted Projects

[Description soundbastlerlive 2023-10-17 13:02:33 UTC]

OpenRC-0.51 breaks cgroupsv2 with various null pointer exceptions for e.g. OpenJDK and docker, resulting in weird errors like in the screenshot.
even reboots on fully upgraded systems don't resolve this.
downgrading to 0.50(-r1 + reboot) fixes it instantly.

Reproducible: Always

Steps to Reproduce:
1. upgrade to openrc-0.51
2. start java service (e.g. wildfly)
Actual Results:  
null pointer exceptions or not-mounted errors regarding cgroups2

[Comment 1 Sam James 2023-10-17 13:03:13 UTC]

Show us some of the example output please?

[Comment 2 soundbastlerlive 2023-10-17 13:03:13 UTC]

Created attachment 873043 [details]
java error for cgroupsv2 with openrc-0.51

[Comment 3 Sam James 2023-10-17 13:15:06 UTC]

(In reply to soundbastlerlive from comment #2)
> Created attachment 873043 [details]
> java error for cgroupsv2 with openrc-0.51

The OpenRC maintainer is blind and will not be able to read a screenshot.

[Comment 4 soundbastlerlive 2023-10-17 16:42:16 UTC]

I used OCR, as I don't have a broken machine at hand right now:

failed to send shutdown request, sending SIGTERM instead: Exception in thread "main" java.lang.NullPointerException at jdk.internal.platform.cgroupv2.CgroupV2Subsystem.getInstance (CgroupV2Subsystem.java:81)
at jdk.internal.platform.CgroupsubsystemFactory.create (CgroupSubsystemFactory.java:108)
at jdk.internal.platform.CgroupMetrics.getInstance (CgroupMetrics.java:173)
at jdk.internal.platform. SystemMetrics.instance (SystemMetrics.java:29)
at jdk.internal.platform.Metrics.systemMetrics (Metrics.java:57)
at jdk.internal.platform.Container.metrics (Container.java:41)
at sun.management.OperatingSystemImpl.<init>(OperatingSystemImpl.java:182)
at sun.management.Management FactoryHelper.getOperatingSystemMXBean (Management FactoryHelper.java:106) java.lang.management.PlatformComponent$7.getMXBeans (PlatformComponent.java:157)
at java.lang.management.PlatformComponent.getMXBeans (PlatformComponent.java:377)
at java.lang.management.ManagementFactory.getPlatformMBeanServer (ManagementFactory.java:472) at org.jboss.modules.ModuleLoader$RealMBeanReg$1.run (ModuleLoader.java:1258)
at org.jboss.modules.ModuleLoader$RealMBeanReg$1.run (ModuleLoader.java:1256)
at java.security.AccessController.doPrivileged (Native Method)
at org.jboss.modules.ModuleLoader$RealMBeanReg.<init>(ModuleLoader.java:1256)
at org.jboss.modules.ModuleLoader$TempMBeanReg.installReal (ModuleLoader.java:1240)
at org.jboss.modules.ModuleLoader.installMBeanServer (ModuleLoader.java:273) at org.jboss.modules.Main.main (Main.java:605)

[Comment 5 Joakim Tjernlund 2023-10-18 08:30:33 UTC]

Guess, this chnage broke it: change default RC_CGROUP_MODE to unified 
https://github.com/OpenRC/openrc/commit/965de92b37cbe8d8670f6cc956e1d10677551e19

[Comment 6 Forza 2023-10-18 16:44:18 UTC]

Is this truly a fault in openrc, or in how 'wildfly' works? I'm using java/openjdk with cgroups v2 since years. Not that this means much in itself, just that it might be worth finding the real root cause.

[Comment 7 soundbastlerlive 2023-10-19 11:53:16 UTC]

I don't know if it's a "fault" in the first place, maybe just some default changed. But that's just a matter of definition I guess.
I'm on vacation and will try next week upgrading and setting that mode to see if that helps.

[Comment 8 Forza 2023-10-19 14:19:27 UTC]

That change did enable cgroups by default, so definitely could be a reason for this problem to surface.

[Comment 9 Sam James 2023-11-12 06:51:53 UTC]

It's possible that the fix in bug 916964 will fix this too.

[Comment 10 Florian Schmaus 2023-11-13 08:45:11 UTC]

Reminds be of bug 894066.

[Comment 11 Sam James 2024-01-03 12:07:47 UTC]

(In reply to Sam James from comment #9)
> It's possible that the fix in bug 916964 will fix this too.

Did 0.52 fix this?

[Comment 12 soundbastlerlive 2024-01-03 17:20:01 UTC]

0.52.1 works for our java cgroup issue (like 0.50-r1 before), however our docker machines still get broken from this (reverting to 0.50-r1 fixes it):

time="2024-01-03T18:18:45.735319186+01:00" level=error msg="stream copy error: reading from a closed fifo"
time="2024-01-03T18:18:45.759443387+01:00" level=error msg="failed to start container" container=fd4756aec98394dc2221069bb236bf525fbb82e025614eebc24a36bf2c973b42 error="failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: unable to apply cgroup configuration: cannot enter cgroupv2 \"/sys/fs/cgroup/docker\" with domain controllers -- it is in domain threaded mode: unknown"

[Comment 13 Forza 2024-01-03 19:15:17 UTC]

Is perhaps the issue that docker tries to use the same cgroup name as the docker init.d service script?

[Comment 14 Sam James 2024-01-04 00:05:20 UTC]

(In reply to soundbastlerlive from comment #12)
> 0.52.1 works for our java cgroup issue (like 0.50-r1 before), however our
> docker machines still get broken from this (reverting to 0.50-r1 fixes it):
> 

Thanks. I think the Docker issue is what Forza is referring to -- see https://github.com/OpenRC/openrc/issues/680. I've pinged the OpenRC maintainer on IRC about merging that.

[Comment 15 Sam James 2024-01-12 18:47:13 UTC]

commit 78dd21dde6daa6e69922552c360d20a9b78c9cc0
Author: William Hubbs <[e-mail]>
Date:   Fri Jan 12 11:51:06 2024 -0600

    sys-apps/openrc: add 0.53

    Signed-off-by: William Hubbs <[e-mail]>

commit 5e47369c3a740dfde792a7013f3ed120e3ac793e
Author: William Hubbs <[e-mail]>
Date:   Thu Nov 16 23:51:37 2023 -0600

    sys-apps/openrc: add 0.52.1

    Signed-off-by: William Hubbs <[e-mail]>