On my system libvirt-6.10.0 intermittently is unable to destroy lxc containers. It fails with % virsh --connect lxc:///system destroy test error: Failed to destroy domain test error: internal error: failed to get cgroup backend for 'pathOfController' It works at first, but when destroying one of the containers fails, it fails consistently. As a first guess, I have set a high kernel pid limit via sysctl: kernel.pid_max = 4194304 Maybe that's the problem.
Correction: This has nothing to do with the kernel.pid_max setting and is actually reproducible (also with libvirt-6.9*)
Matthias, are you using cgroups v1 or v2? With systemd? I'm running v1 with openrc and can't reproduce.
This is cgroup v2 with sytemd.
Created attachment 679524 [details] emerge --info
And I did not encounter this problem with libvirt-6.9 and cgroupv1 (and systemd).
This bug is present in 7.0.0 and 7.1.0 as well.
This bug is present in 7.2.0 as well.
Matthias, I'm sorry to hear that. Can you please attach debug logs? https://libvirt.org/kbase/debuglogs.html
Created attachment 700038 [details] libvirtd debug output
Created attachment 700041 [details] lxc container logfile
(In reply to Michal Privoznik from comment #8) > Matthias, I'm sorry to hear that. Can you please attach debug logs? > > https://libvirt.org/kbase/debuglogs.html Attached. By the way, I can reliably trigger this issue on Debian bullseye (which mounts cgroupv2 only) as well. I have attached the lxc log for startup of a container named "amd64", as well as the debug output for libvirtd for the following command: % virsh --connect lxc:///system destroy amd64 error: Failed to destroy domain 'amd64' error: internal error: failed to get cgroup backend for 'pathOfController'
Created attachment 700050 [details] container xml definition
Thank you, I was able to reproduce. The problem is with how libvirt removes nested controllers. I've posted patch here: https://listman.redhat.com/archives/libvir-list/2021-April/msg00756.html
Merged upstream as: ea7d0ca37c vircgroup: Fix virCgroupKillRecursive() wrt nested controllers a0815484b1 vircgroupbackend: Extend error messages in VIR_CGROUP_BACKEND_CALL() edce157f11 vircgroup: Debug print all arguments of virCgroupKillRecursiveInternal() v7.2.0-228-gea7d0ca37c
(In reply to Michal Privoznik from comment #14) > Merged upstream as: > > [...] Very nice! I will prepare a revision bump for the Gentoo package in a minute.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ca152da5d31120472ffce4b687550a5454b2a11f commit ca152da5d31120472ffce4b687550a5454b2a11f Author: Michal Privoznik <mprivozn@redhat.com> AuthorDate: 2021-04-20 07:36:45 +0000 Commit: Matthias Maier <tamiko@gentoo.org> CommitDate: 2021-04-20 14:09:58 +0000 app-emulation/libvirt: Allow destroy of LXC containers again The original problem was fixed upstream as: ea7d0ca37c vircgroup: Fix virCgroupKillRecursive() wrt nested controllers and the commit will be part of the upcoming 7.3.0 release. However, the bug is so critical that the fix deserves to be backported to all supported releases. Please note, that for libvirt-7.2.0 I'm also dropping the code under src_install() that's supposed to fix docdir for ebuilds with revision number. This fixup is not needed because as of cc20e6298b7217f4b0eab3c50078257fd77ebb1b the docdir is put correctly onto meson's cmd line. I'm doing these two changes to avoid necessary revision number bump. Closes: https://bugs.gentoo.org/761721 Closes: https://github.com/gentoo/gentoo/pull/20468 Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Signed-off-by: Matthias Maier <tamiko@gentoo.org> ...ix-virCgroupKillRecursive-wrt-nested-cont.patch | 189 +++++++++++++++++++++ ...irt-7.0.0-r1.ebuild => libvirt-7.0.0-r2.ebuild} | 1 + ...ibvirt-7.1.0.ebuild => libvirt-7.1.0-r1.ebuild} | 1 + ...ibvirt-7.2.0.ebuild => libvirt-7.2.0-r1.ebuild} | 7 +- 4 files changed, 192 insertions(+), 6 deletions(-)
