This bug should probably be assigned to Blueness. GRS tries to create a new cgroup tree in /sys/fs/cgroup . Systemd mounts /sys/fs/cgroup as read only. It's relatively simple to work around this. remount -o remount,rw /sys/fs/cgroup GRS would benefit from code that detects this situation and gives the users instructions they can follow.
(In reply to Michael Jones from comment #0) > This bug should probably be assigned to Blueness. > > GRS tries to create a new cgroup tree in /sys/fs/cgroup . Systemd mounts > /sys/fs/cgroup as read only. > > It's relatively simple to work around this. > > remount -o remount,rw /sys/fs/cgroup > > > > > GRS would benefit from code that detects this situation and gives the users > instructions they can follow. yeah i've never run GRS on a systemd system. What can go wrong with that remount.
(In reply to Michael Jones from comment #0) > > remount -o remount,rw /sys/fs/cgroup > @systemd folks, does anyone know if remounting cgroup rw is bad? GRS [1] is a suite for building releases (anything from stage1/2/3 to full blown desktops) and it makes use of cgroups to collect together processes associated with some chroot, much the way systemd-nspawn does. To do so, it needs /sys/fs/cgroup to be read/write. Is there any reason not to do so? [1] https://wiki.gentoo.org/wiki/Project:RelEng_GRS
(In reply to Anthony Basile from comment #2) I don't fully understand why systemd mounts it readonly, but I think it should be fine to remount rw for this use case.
A possibility to consider is to optionally use systemd-nspawn, instead of chroot + cgroups on systems that use systemd. https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html
(In reply to Michael Jones from comment #4) > A possibility to consider is to optionally use systemd-nspawn, instead of > chroot + cgroups on systems that use systemd. > > https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html can you suggest a patch for GRS?(In reply to Michael Jones from comment #4) > A possibility to consider is to optionally use systemd-nspawn, instead of > chroot + cgroups on systems that use systemd. > > https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html i guess i can make it a configure option, either run the scripts in the build system via chroot+cgroups or via systemd-nspawn. right now the chroot+cgroups system provides grsrun with the set of tasks running in the chroot so that its easy to clean up, something that catalyst can't do and which drives me crazy! if its possible to use systemd-nspawn to do the same cleanup then i'll just have a switch in the code to use one or the other. i'll probably have to add that over the summer when i have time. in the mean time, i think i can just add a check if /sys/fs/cgroup is mounted ro and remount it rw if it is. lets leave this bug open to remind me to consider using systemd-nspawn.
I'll see if I can come up with a patch for grs that does what I suggested. systemd-nspawn automatically runs all processes happening in the "chroot" of the nspawn environment in a cgroup. When the process that systemd-nspawn started in the chroot exits, all remaining processes are cleaned up. Do you have multiple independent groups of processes that you track with cgroups? or just one group per build environment? If you track more than one group per build environment, maybe systemd-nspawn isn't a good fit.
nspawn will create its own cgroups, and I believe it will clean up everything in the container when the process it spawns dies (including mounts, etc). I'm pretty sure that you can just sigterm nspawn and get it to do its own cleanup as well.
(In reply to Michael Jones from comment #6) > I'll see if I can come up with a patch for grs that does what I suggested. > > systemd-nspawn automatically runs all processes happening in the "chroot" of > the nspawn environment in a cgroup. When the process that systemd-nspawn > started in the chroot exits, all remaining processes are cleaned up. > > Do you have multiple independent groups of processes that you track with > cgroups? or just one group per build environment? If you track more than one > group per build environment, maybe systemd-nspawn isn't a good fit. GRS runs multiple concurrent build environments, but each one has only one group of tasks. if you send grsrun SIGINT or SIGTERM, it will first send SIGTERM and then SIGKILL to all processes in the build env task group so that i can cleanup the chroot. i think systemd-nspawn might be able to do that.
(In reply to Richard Freeman from comment #7) > nspawn will create its own cgroups, and I believe it will clean up > everything in the container when the process it spawns dies (including > mounts, etc). I'm pretty sure that you can just sigterm nspawn and get it > to do its own cleanup as well. sounds like i wrote my own "poor mans" version of nspawn on top of libcgroup.
(In reply to Anthony Basile from comment #8) > (In reply to Michael Jones from comment #6) > > I'll see if I can come up with a patch for grs that does what I suggested. > > > > systemd-nspawn automatically runs all processes happening in the "chroot" of > > the nspawn environment in a cgroup. When the process that systemd-nspawn > > started in the chroot exits, all remaining processes are cleaned up. > > > > Do you have multiple independent groups of processes that you track with > > cgroups? or just one group per build environment? If you track more than one > > group per build environment, maybe systemd-nspawn isn't a good fit. > > GRS runs multiple concurrent build environments, but each one has only one > group of tasks. if you send grsrun SIGINT or SIGTERM, it will first send > SIGTERM and then SIGKILL to all processes in the build env task group so > that i can cleanup the chroot. i think systemd-nspawn might be able to do > that. I agree. Just launch one nspawn instance per environment. In addition to being a chroot and using cgroups, nspawn launches processes by default in a process/mount namespace. You can also choose to put it in a network namespace (if it doesn't need network access you could isolate it from the network trivially). With the mount namespace you don't have to worry about any kind of cleanup if anything crashes, and of course with a process namespace there is more isolation (catalyst could still see the processes inside the container, but not the reverse). The magic invocation is typically something like this: systemd-nspawn -D path/to/chroot --bind /usr/portage --bind /foo/distfiles:/var/cache/distfiles/ /bin/bash That will spawn /bin/bash in a container that behaves like a chroot in path/to/chroot, and will set up two bind mounts from the host into the chroot. I don't know if you really need bind mounts, and you can just omit that or add as many as you need. The man page is actually pretty good, and if you google for an nspawn guide you'll probably find my blog pretty high in the pagerank. :)
