Created attachment 746112 [details] output of nft list ruleset after loading the original file I incrementally setup nftables rules on a machine with a default policy to drop packages. The machine runs bind, apache2, sshd, minidlna, mpd, postgres, CUPS and mosquitto. While editing the rules everything works fine. I can flush the ruleset, edit, load again and I can see the effects taking place. For example omitting TCP port 631 yields log messages that incoming traffic was denied. Adding it again and CUPS works fine. With a final working ruleset I rebooted the machine. After it started successfully I cannot access ssh anymore: connection timeout. There are no log messages about denied traffic. I can fix this by simply reloading my ruleset through nft directly. * iptables is not running anymore, removed completely from the kernel * diff between the two variants - after boot and after nft -f - shows now obvious differences * changing order of init scripts has no effect (nftables is in run level boot, net.* in default, switched this to even nftables in default with depend net) * the netdev rule is loaded successfully even before net.enp8s0 is there (using string comparison/ interface name does not rely on device) Could it be nft-wrapping through /etc/init.d/nftables shipped with the package?
Created attachment 746115 [details] output of nft list ruleset after boot/ startup
Created attachment 746121 [details] original nftables configuration
I could track it down to a single rule. In the original file there is the rule: tcp flags & (fin|syn|rst|ack) != syn ct state new counter drop After /etc/init.d/nftables save this is translated to: tcp flags 0x2 / 0x1,0x2,0x4,0x10 ct state 0x8 counter packets 0 bytes 0 drop Removing this line and loading this with /etc/init.d/nftables restart enables connections to sshd. Also replacing the integer-variant with the verbose original (and re-loading) enables ssh connections. Reverting this to the integer-variant gives connection timeout again. Interestingly logging seems to be broken completely then. There is a rule in the same table's chain: log prefix "[nftables] Inbound Denied: " counter drop Neither with the integer variant nor the verbose original tcp flag rule messages appear in the logs regarding dropped packages.
This happens with version 0.9.9 (current stable) and boils down to something wrong with the numeric mode (-n) when saving the rule set. Original verbose expression is: tcp flags & (fin|syn|rst|ack) != syn ct state new counter drop Saving this through /etc/init.d/nftables turns it with 0.9.9 into: tcp flags 0x2 / 0x1,0x2,0x4,0x10 ct state 0x8 counter packets 0 bytes 0 drop Saving the exactly same verbose expression with 1.0.0 yields: tcp flags != 0x2 / 0x1,0x2,0x4,0x10 ct state 0x8 counter packets 0 bytes 0 drop And suddenly it works. So there are basically two fixes: 1. Upgrade to 1.0.0 (with accept_keywords or simply wait until stabelized) 2. Remove -n from SAVE_OPTIONS in /etc/conf.d/nftables
Hey! Can you provide a copy of /var/lib/nftables/rules-save That file contains the ruleset that is loaded when the service is restarted. I'm trying to reproduce your issue but I only have issues with the ingress filtering hook (I think because I may have it disabled on the kernel). type filter hook ingress device "enp8s0" priority -500; policy accept; All the other rules load and are saved correctly.
Also if you don't want nftables to rewrite your ruleset and drop your comments you can check the option SAVE_ON_STOP on /etc/conf.d/nftables.
I wrote to nftables mailing list to get some details on the issue. It is only happening in 0.9.9 and seems to be fixed in 1.0.0. And it seems to be irrelevant where the rule is defined and what is around it. It might be related to the family inet. I'll try it with ip resp. ip6, too. But it has to be something with the TCP-flags (3rd rule) and numeric mode (-n): table inet firewall { chain inbound { type filter hook input priority 0; policy drop; ct state vmap { established : accept, related : accept, invalid : drop } tcp flags & (fin|syn|rst|ack) != syn ct state new counter drop } } Saving this or exporting it with nft -n list ruleset yields with 1.0.0: table inet firewall { chain inbound { type filter hook input priority 0; policy drop; ct state vmap { 0x1 : drop, 0x2 : accept, 0x4 : accept } tcp flags != 0x2 / 0x1,0x2,0x4,0x10 ct state 0x8 counter packets 0 bytes 0 drop } With 0.9.9 the 3rd line has a different operator (default operator which is eq): tcp flags 0x2 / 0x1,0x2,0x4,0x10 ct state 0x8 counter packets 0 bytes 0 drop This means the exact opposite. SAVE_ON_STOP would be another option, you're absolutely right. While moving towards a working configuration I found it quite handy to use the roundtrip through the init script. It'd be also more secure to not save on stop therefore booting with the approved rule set instead of a potentially hacked.
Well security-wise you may need something stronger than SAVE_ON_STOP to avoid access after a compromise. That said, what you are commenting is a valid bug and if it is solved on 1.0.0 you may want to add that version to accept_keywords.
#790014 is likely related (although it impacts a different configuration directive).
(In reply to Francisco Blas Izquierdo Riera from comment #8) > […] > > That said, what you are commenting is a valid bug and if it is solved on > 1.0.0 you may want to add that version to accept_keywords. Did so=works. But with 0.9.9 being stable the bug persists. I already pointed it out in the wiki[1]. And I enqueue myself to the line of critics of using numeric mode for saving. With Bug#790014 this is the second time it broke this way. Is it now confirmed? [1] https://wiki.gentoo.org/wiki/Nftables#Restart_of_nftables_or_reboot_cause_blocked_connections
(In reply to Francisco Blas Izquierdo Riera from comment #8) > Well security-wise you may need something stronger than SAVE_ON_STOP to > avoid access after a compromise. > > That said, what you are commenting is a valid bug and if it is solved on > 1.0.0 you may want to add that version to accept_keywords. Seems like we should stable a newer version then?
Indeed Sam! Since all the deps are the same I'd recommend we stabilize 1.0.0 @base-system, this has been in ~arch for more than one month already, if you want to wait please say so before this saturday, otherwise I'll request stabilization then.
I am fine stabilizing 1.0.0, AFAIK there have been no new bugs against it.
Bug with tcp flags != confirmed on mailing list: https://marc.info/?l=netfilter&m=163528872200611&w=2
Works stable now (1.0.2).
