systemd installs /usr/lib/sysctl.d/coredump.conf which causes it to intercept all coredumps and store them in its journal. Unfortunately there is not yet a sane way of getting them back *out* of that journal, so you can run gdb on them and the like (at least not that I can find in "man journalctl", or that freenode's #systemd knows about). Until such tools materialize the ebuild should not install /usr/lib/sysctl.d/coredump.conf. Just to clarify: it currently seems to intercept *all* coredumps, not just ones produced by daemons systemd is managing. This interferes greatly with development work until you've figured out how to flip it back off (sysctl kernel.core_pattern=core).
What does upstream say? Why they are installing this file? Also, I think that journal actually stores truncated coredumps or something like that. I tried to get it once using journalctl exports (binary and JSON), and in both cases the resulting file wasn't usable to gdb.
I've filed https://bugs.freedesktop.org/show_bug.cgi?id=54288 .
My upstream bug report is not getting any attention. I filed that after I'd asked freenode's #systemd how to deal with these coredumps, and nobody there at that time really had a good way. Also, fedora (which AFAIK is the most systemd-oriented distro right now) disables this feature, see http://pkgs.fedoraproject.org/cgit/systemd.git/tree/systemd.spec#n276 : # To avoid making life hard for Rawhide-using developers, don't package the # kernel.core_pattern setting until systemd-coredump is a part of an actual # systemd release and it's made clear how to get the core dumps out of the # journal. /usr/bin/rm -f %{buildroot}%{_prefix}/lib/sysctl.d/coredump.conf If you're unwilling to remove this file can you at least tell me how to extract usable coredumps back out of the journal? It's getting a bit annoying to trigger crashes twice (because I keep forgetting to remove this again after systemd upgrades).
(In reply to comment #3) > My upstream bug report is not getting any attention. I filed that after I'd > asked freenode's #systemd how to deal with these coredumps, and nobody there > at that time really had a good way. Also, fedora (which AFAIK is the most > systemd-oriented distro right now) disables this feature, see > http://pkgs.fedoraproject.org/cgit/systemd.git/tree/systemd.spec#n276 : > > # To avoid making life hard for Rawhide-using developers, don't package the > # kernel.core_pattern setting until systemd-coredump is a part of an actual > # systemd release and it's made clear how to get the core dumps out of the > # journal. > /usr/bin/rm -f %{buildroot}%{_prefix}/lib/sysctl.d/coredump.conf > > If you're unwilling to remove this file can you at least tell me how to > extract usable coredumps back out of the journal? It's getting a bit > annoying to trigger crashes twice (because I keep forgetting to remove this > again after systemd upgrades). Thanks for doing the research. Could you check whether renaming the file to 'coredump.conf.disabled' or something like that disables it as well?
(In reply to comment #4) > Could you check whether renaming the file to > 'coredump.conf.disabled' or something like that disables it as well? Yes, it checks for *.conf. The directories it checks are /etc/sysctl.d, /run/sysctl.d, /usr/local/lib/sysctl.d and /usr/lib/sysctl.d (and possibly /lib/sysctl.d), so if you're going to rename the file so users can enable it by renaming it back I'd suggest moving it from /usr/lib/sysctl.d to /etc/sysctl.d, so users don't have to mess with files in a supposedly-readonly directory.
(In reply to comment #5) > (In reply to comment #4) > > Could you check whether renaming the file to > > 'coredump.conf.disabled' or something like that disables it as well? > > Yes, it checks for *.conf. The directories it checks are /etc/sysctl.d, > /run/sysctl.d, /usr/local/lib/sysctl.d and /usr/lib/sysctl.d (and possibly > /lib/sysctl.d), so if you're going to rename the file so users can enable it > by renaming it back I'd suggest moving it from /usr/lib/sysctl.d to > /etc/sysctl.d, so users don't have to mess with files in a > supposedly-readonly directory. Yes, that's exactly what I was considering.
Fixed in -194.
-195 introduced a tool (systemd-coredumpctl) to grab coredumps, so I have re-enabled storing them in journal. Is that ok with you?
It looks like access control is tied to who can read the journal. That means: users in the "adm" group can read all coredumps (also coredumps generated by executables that ran as other users), users not in the "adm" group cannot read coredumps. That seems like a regression over the kernel-provided default behavior (core file owned by the same user as the crashed process, 600 permissions), although it's not a huge deal for my dev system. Note systemd-coredumpctl doesn't tell you you need to be in that group, it just tells you there aren't any coredumps. Also note systemd-coredumpctl itself is still not particularly discoverable, although I don't really see how you could improve that. I now wonder what else being in the "adm" group gets you: I believe I added myself to that group when journalctl mentioned "Users in the group 'adm' can see all messages." Coredumps (which will definitely contain sensitive data occasionally) being "messages" is non-obvious. It looks like this only provides convenient access to the most recent coredump for a certain PID or pathname. Access to older coredumps is possible but nontrivial. I guess I'm -0 to leaving this enabled: I'd slightly prefer it disabled but don't really care much now that I can at least dig coredumps back out without having to write code to do so. But I still boggle at what upstream is trying to do here. Is there an advantage to storing coredumps this way that I overlook? If there are no noticable advantages to this (yet) and/or other distros leave it disabled I'd leave this disabled in gentoo too for now. Perhaps tie it to the "vanilla" USE flag?
(In reply to comment #9) > It looks like access control is tied to who can read the journal. That > means: users in the "adm" group can read all coredumps (also coredumps > generated by executables that ran as other users), users not in the "adm" > group cannot read coredumps. That seems like a regression over the > kernel-provided default behavior (core file owned by the same user as the > crashed process, 600 permissions), although it's not a huge deal for my dev > system. Note systemd-coredumpctl doesn't tell you you need to be in that > group, it just tells you there aren't any coredumps. Also note > systemd-coredumpctl itself is still not particularly discoverable, although > I don't really see how you could improve that. > > I now wonder what else being in the "adm" group gets you: I believe I added > myself to that group when journalctl mentioned "Users in the group 'adm' can > see all messages." Coredumps (which will definitely contain sensitive data > occasionally) being "messages" is non-obvious. I guess that's something to report/ask upstream. > It looks like this only provides convenient access to the most recent > coredump for a certain PID or pathname. Access to older coredumps is > possible but nontrivial. > > I guess I'm -0 to leaving this enabled: I'd slightly prefer it disabled but > don't really care much now that I can at least dig coredumps back out > without having to write code to do so. But I still boggle at what upstream > is trying to do here. Is there an advantage to storing coredumps this way > that I overlook? If there are no noticable advantages to this (yet) and/or > other distros leave it disabled I'd leave this disabled in gentoo too for > now. Perhaps tie it to the "vanilla" USE flag? I don't think 'vanilla' fits here. I will switch this back off for now and see what you get with the bug you mentioned before.
I think its time to enable coredumps now. when systemd-coredumpctl is available for coredumps management.
Worked great for me, after activating via sysctl.conf.d and figuring out (incompletly) how choose the dump I want from the list. I found it unsettling to visualize just how many cores I really dump day in and day out -- lots of 'em.
I'll test it on my machine a bit as well. Considering that the ownership issue has been fixed, we may consider enabling it.
I use systemd user-session services. When one of the user-session services dumps its core, it is stored with "-rw-r-----+ 1 root root" in /var/lib/systemd/coredump, so I'd say this looks good so far permission-wise. The ACL of those coredumps is set to allow the user under which the user-session services was running to read the coredump: getfacl /var/lib/systemd/coredump/core.drkonqi.1000.83f053070797496990bb556c398369c8.6651.1409741642000000.xz getfacl: Removing leading '/' from absolute path names # file: var/lib/systemd/coredump/core.drkonqi.1000.83f053070797496990bb556c398369c8.6651.1409741642000000.xz # owner: root # group: root user::rw- user:elias:r-- group::r-- mask::r-- other::--- It's a bit weird to have the ACL set first to 'user::rw-' and then to 'user:elias:r--', so I get first 'rw' on the dump, to have 'w' then revoked by 'user:elias:r--'. Anyways, the resulting permissions look good to me.
Can you please re-enable this systemd feature at least when the elfutils useflag is enabled?
Please re-enable as coredumpctl(1) works flawlessly. $ ./nullpointer Segmentation fault (core dumped) $ coredumpctl TIME PID UID GID SIG PRESENT EXE Do 2016-04-28 14:33:25 CEST 30156 1000 1000 11 * /tmp/nullpointer $ coredumpctl dump -o coredump PID: 30156 (nullpointer) UID: 1000 (qsx) GID: 1000 (qsx) Signal: 11 (SEGV) Timestamp: Do 2016-04-28 14:33:25 CEST (48s ago) Command Line: ./nullpointer Executable: /tmp/nullpointer Control Group: /user.slice/user-1000.slice/session-c3.scope Unit: session-c3.scope Slice: user-1000.slice Session: c3 Owner UID: 1000 (qsx) Boot ID: 5e3662d58adf43cf9fd8e944bfa549e9 Machine ID: afaa06e99f2eef980113f05e57124dbc Hostname: coruscant.qsuscs.de Coredump: /var/lib/systemd/coredump/core.nullpointer.1000.5e3662d58adf43cf9fd8e944bfa549e9.30156.1461846805000000000000.lz4 Message: Process 30156 (nullpointer) of user 1000 dumped core. More than one entry matches, ignoring rest. $ file coredump coredump: ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from './nullpointer', real uid: 1000, effective uid: 1000, real gid: 1000, effective gid: 1000, execfn: './nullpointer', platform: 'x86_64'
It has been re-enabled in the live ebuild. I will also enable it in systemd 230 once it is released.
This is re-enabled in systemd-230.
