After updating to systemd-249.11 systemd-resolve breaks using the default configuration with "DNSSEC validation failed: missing-key". Reproducible: Always Steps to Reproduce: 1. merge sys-apps/systemd-249.11 2. run `resolvectl query gentoo.org` 3. dns fails Actual Results: After emerging systemd to the latest stable version dns slowly starts breaking down. ``` $ resolvectl query gentoo.org gentoo.org: resolve call failed: DNSSEC validation failed: missing-key ``` ``` $ journalctl --unit systemd-resolved Mar 29 09:59:14 hostname systemd-resolved[1386]: [🡕] DNSSEC validation failed for question . IN DNSKEY: missing-key Mar 29 09:59:14 hostname systemd-resolved[1386]: [🡕] DNSSEC validation failed for question com IN DS: missing-key Mar 29 09:59:14 hostname systemd-resolved[1386]: [🡕] DNSSEC validation failed for question com IN DNSKEY: missing-key Mar 29 09:59:14 hostname systemd-resolved[1386]: [🡕] DNSSEC validation failed for question mozilla.com IN DS: missing-key .... Logs continue in this fashion ``` ``` $ grep -vE '^#' /etc/systemd/resolved.conf [Resolve] ``` Expected Results: DNS resolving continues to work after updating systemd ## Workaround: Adding: "DNSSEC=false" to "/etc/systemd/resolved.conf" resulted in a working system. ## Info sys-apps/systemd use flags: +abi_x86_32 +acl +cgroup-hybrid +gcrypt +kmod +lz4 +pam +pcre +policykit +resolvconf +seccomp +sysv-utils +zstd
emerge.log shows that it merged glibc right before systemd, which I now think is the issue... ``` emerge.log: Tue Mar 29 09:59:17 2022 >>> sys-libs/glibc-2.34-r10 Tue Mar 29 09:59:23 2022 >>> sys-devel/binutils-config-5.4.1 Tue Mar 29 10:00:44 2022 >>> sys-apps/systemd-249.11 Tue Mar 29 10:00:51 2022 >>> dev-python/flit_core-3.7.1 systemd-resolve log: Mar 29 09:59:14 hostname systemd-resolved[1386]: [🡕] DNSSEC validation failed for question ``` After trying 250.4, and 9999, both had the same problems, so i reverted back to 249.9 and rebooted. Same issue. DNSSEC failing notifications. The timings nearly match up exactly for glibc issues rather than the systemd install... (see attached emerge.log)
Created attachment 768103 [details] emerge log with exact timings
Hi I have the very same behaviour of OP. It seems related to upstream systemd issue 10579 (can't post links) which is present since 2018. I suspected that glibc switched from its own dns resolver to systemd in 2.34 but I found nothig in the release notes.
Based on comment 1, this is probably not a regression in systemd, but rather some interaction with a library upgrade.
Is this still a problem with recent systemd releases?
Hi Mike, Thanks for reminding me. (In reply to Mike Gilbert from comment #5) > Is this still a problem with recent systemd releases? I masked all systemd versions after 249.9 so i could have a working system. I have since updated to systemd-253.3-r1 and it successfully works without the workaround listed above.
I spoke too soon, but I have successfully replicated the issue on my system with the latest systemd version. (systemd-253.6) The problem seems to relate to using a DNS resolver that is "outside" my network, that does not support DNSSEC. resolvectl doesn't seem to figure out that it isn't supported and as such continues to try? (I'm guessing here) Replication: 1. Pick a random DNS server from OpenNIC: servers.opennic.org 2. Set it as your DNS resolver: sudo resolvectl dns enp0 168.138.8.38 3. Show the status: resolvectl status enp0 ``` Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6 Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=allow-downgrade/supported DNS Servers: 168.138.8.38 ``` 3.1 See the DNSSEC=allow-downgrade/supported 4. Query Something: resolvectl query bugs.gentoo.org 4.1 `bugs.gentoo.org resolve call failed: DNSSEC validation failed: missing-key` 5. Check the status again: resolvectl status enp0 5.1 Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=allow-downgrade/supported 5.2 Still shows as supported, even though the server actually doesn't support it 6. Change back to your local router dns: sudo resolvectl dns enp0 192.168.0.1 && resolvectl status enp0 6.1 it will show DNSSEC=allow-downgrade/supported until a DNS request is done 7. resolvectl query bugs.gentoo.org && resolvectl status enp0 7.1 correctly resolves and shows that DNSSEC=allow-downgrade/unsupported Proof: Use OpenDNS' 208.67.222.222 and you can see that dnssec passes and no issues arise. ---------- Work around: don't use a Non-DNSSEC enabled external resolver.