Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 501708

Summary: net-misc/openssh - sshd.socket won't trigger ssh host key generation
Product: Gentoo Linux Reporter: Pavel Šimerda <code>
Component: Current packagesAssignee: Gentoo systemd Team <systemd>
Status: CONFIRMED ---    
Severity: normal CC: base-system, gentoo
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=831762
Whiteboard:
Package list:
Runtime testing required: ---

Description Pavel Šimerda 2014-02-18 18:52:16 UTC
The classic sshd.service will ensure host keys are generated by calling 'ssh-keygen -A' using ExecStartPre, but the sshd@.service triggered by sshd.socket won't do that. I don't know how well ExecStartPre with @-services. There's another advantage of not spawning a new process when the keys are already there.

sshd_at.service:

@@ -1,6 +1,7 @@
 [Unit]
 Description=OpenSSH per-connection server daemon
-After=syslog.target auditd.service
+Wants=sshd-keygen.service
+After=syslog.target auditd.service sshd-keygen.service
 
 [Service]
 ExecStart=-/usr/sbin/sshd -i -e

The classic sshd.service could use it as well.

sshd.service:

 [Unit]
 Description=OpenSSH server daemon
-After=syslog.target network.target auditd.service
+Wants=sshd-keygen.service
+After=syslog.target network.target auditd.service sshd-keygen.service
 
 [Service]
-ExecStartPre=/usr/bin/ssh-keygen -A
 ExecStart=/usr/sbin/sshd -D -e
 ExecReload=/bin/kill -HUP $MAINPID

And this is how the sshd-keygen service could look like.

sshd-keygen.service:

[Unit]
Description=OpenSSH Server Key Generation
ConditionPathExists=|!/etc/ssh/ssh_host_rsa_key
ConditionPathExists=|!/etc/ssh/ssh_host_dsa_key

[Service]
ExecStart=/usr/bin/ssh-keygen -A
Type=oneshot
Comment 1 Pavel Šimerda 2014-02-18 19:04:02 UTC
Also, I'm not entirely sure but maybe the 'ssh-keygen.service' dependencies should go to 'sshd.socket' instead of 'sshd@.service'.
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-02-18 19:12:14 UTC
Honestly, I'd drop the 'Wants=' completely, and instead use 'Also=' in [Install] to enable the keygen on boot whenever any of SSH services is enabled. That should be cleaner than auto-pulling it when SSH is started.
Comment 3 Pavel Šimerda 2014-02-21 13:47:40 UTC
(In reply to Michał Górny from comment #2)
> Honestly, I'd drop the 'Wants=' completely, and instead use 'Also=' in
> [Install] to enable the keygen on boot whenever any of SSH services is
> enabled. That should be cleaner than auto-pulling it when SSH is started.

I think Unit.Wants is much cleaner, as it's a dependency for the SSH daemon and not a separate service. Install.Also will work only under some specific circumstances, wheres Unit.Wants expresses exactly what the relation between sshd and sshd-keygen is.
Comment 4 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-02-21 14:41:46 UTC
(In reply to Pavel Šimerda (pavlix) from comment #3)
> (In reply to Michał Górny from comment #2)
> > Honestly, I'd drop the 'Wants=' completely, and instead use 'Also=' in
> > [Install] to enable the keygen on boot whenever any of SSH services is
> > enabled. That should be cleaner than auto-pulling it when SSH is started.
> 
> I think Unit.Wants is much cleaner, as it's a dependency for the SSH daemon
> and not a separate service. Install.Also will work only under some specific
> circumstances, wheres Unit.Wants expresses exactly what the relation between
> sshd and sshd-keygen is.

You are wrong. sshd needs *keys*, not a random key generation service that tries to generate new keys every time sshd is started or -- as requested here -- every time someone connects to ssh...
Comment 5 Mike Gilbert gentoo-dev 2014-02-21 15:31:26 UTC
I think we could avoid that by setting RemainAfterExit=true so that it is only invoked once per system boot.

However, since this is really a workaround for not being able to generate keys in pkg_postinst, the [Install] idea should be fine.
Comment 6 Pavel Šimerda 2014-02-21 17:29:08 UTC
(In reply to Michał Górny from comment #4)
> sshd needs *keys*, not a random key generation service

Could you elaborate about the meaning of 'random' in this sentence?

> that tries to generate new keys every time sshd is started

When ConditionPathExists is in use, systemd actually checks for existence of the keys to avoid starting the command if it's not necessary. Also, with RemainAfterExit, it won't be AFAIK triggered on sshd restarts (only tested with OpenRC, not systemd).

> or -- as requested here -- every time someone connects to ssh...

I certainly didn't request to run the key generator for every SSH connection.

For sshd.service service, the only time when running the key generator is even considered, is when the service is requested to start at boot time or by the user. For sshd@.service, I suggested in comment #1 to put the Wants to the sshd.socket instead of sshd@.service to achieve the very same effect.

(In reply to Mike Gilbert from comment #5)
> I think we could avoid that by setting RemainAfterExit=true so that it is
> only invoked once per system boot.

Yes, RemainAfterExit=true is a good idea.

> However, since this is really a workaround for not being able to generate
> keys in pkg_postinst,

That's not entirely true from the point of a system administrator or vendor who has images of filesystems with all identification and security data deleted. In that case it serves to generate the keys when they are needed and missing.

> the [Install] idea should be fine.

From http://www.freedesktop.org/software/systemd/man/systemd.unit.html:

"This section is not interpreted by systemd(1) during runtime."

So please be careful about the logic, here. I don't know how exactly it plays with the systemd presets but I can ask the systemd developers.

Note that if a user wanted to explicitly avoid running the key generation service entirely, he can still mask the sshd-keygen.service and the sshd.service will still work.

I don't see a reason to search for new tools for something that's properly handled by the Wants option.
Comment 7 Pavel Šimerda 2014-05-08 11:15:34 UTC
Hi,

bringing some new information for the bug report.

1) We need to avoid the [Install] section entirely when we're talking about runtime dependencies. 

Rationale:

Imagine a system where sshd is not enabled (neither sshd.service nor sshd.socket) and so it's not run at boot time. Then a user wants to have systemd available with one of the following commands:

systemctl start sshd.service

systemctl start sshd.socket

Both of those commands ask systemd to start a unit file and therefore the Install section is not being used at all in this process per documentation:

"This section is not interpreted by systemd(1) during runtime."

http://www.freedesktop.org/software/systemd/man/systemd.unit.html

2) RemainAfterExit=true (as pointed out by Mike) is *important* to keep track of the fact that ssh-keygen has been already successfully run.

3) I personally would prefer if sshd.socket depended on sshd-keygen.service so that key generation is performed at boot. An alterative is to have sshd@.service depend on it to perform key generation at first connected client and rely on #2 to ensure it's not run repeatedly.

4) Ordering after auditd.service is redundant. There's an implicit ordering of sshd.* on basic.target which is ordered after sysinit.target and auditdi.service should be simply ordered before sysvinit.target (true for Fedora, didn't check for Gentoo, yet).

5) Ordering after network.target IMO incorrect. Only network-online.target says that network connectivity is already established and SSH shouldn't need that either and should use IP_FREEBIND if needed.

6) Ordering after syslog.target is redundant and is no longer encouraged by systemd upstream.

http://www.freedesktop.org/wiki/Software/systemd/syslog/
Comment 8 Mike Gilbert gentoo-dev 2014-05-08 16:09:22 UTC
Another issue that I thought of: When you upgrade OpenSSH, occasionally support for a new key format is added. Therefore, we would not want to limit key generation to once per boot.

Given that, I think we should probably just add ExecStartPre=/usr/bin/ssh-keygen -A to sshd@.service. The overhead of running that command when the keys already exist is trivial.
Comment 9 Markos Chandras (RETIRED) gentoo-dev 2014-05-08 19:43:10 UTC
arch linux does the following[1]

sshdgenkeys.service:

[Unit]
Description=SSH Key Generation
ConditionPathExists=|!/etc/ssh/ssh_host_key
ConditionPathExists=|!/etc/ssh/ssh_host_key.pub
ConditionPathExists=|!/etc/ssh/ssh_host_rsa_key
ConditionPathExists=|!/etc/ssh/ssh_host_rsa_key.pub
ConditionPathExists=|!/etc/ssh/ssh_host_dsa_key
ConditionPathExists=|!/etc/ssh/ssh_host_dsa_key.pub
ConditionPathExists=|!/etc/ssh/ssh_host_ecdsa_key
ConditionPathExists=|!/etc/ssh/ssh_host_ecdsa_key.pub
ConditionPathExists=|!/etc/ssh/ssh_host_ed25519_key
ConditionPathExists=|!/etc/ssh/ssh_host_ed25519_key.pub

[Service]
ExecStart=/usr/bin/ssh-keygen -A
Type=oneshot
RemainAfterExit=yes




sshd@.service
[Unit]
Description=OpenSSH Per-Connection Daemon
After=sshdgenkeys.service

[Service]
ExecStart=-/usr/bin/sshd -i
StandardInput=socket
StandardError=syslog




sshd.socket
[Unit]
Conflicts=sshd.service
Wants=sshdgenkeys.service

[Socket]
ListenStream=22
Accept=yes

[Install]
WantedBy=sockets.target



sshd.service:
[Unit]
Description=OpenSSH Daemon
Wants=sshdgenkeys.service
After=sshdgenkeys.service
After=network.target

[Service]
ExecStart=/usr/bin/sshd -D
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=always

[Install]
WantedBy=multi-user.target

[1] https://www.archlinux.org/packages/core/i686/openssh/download/

Can we do something similar?
Comment 10 Pavel Šimerda 2014-05-09 10:57:06 UTC
(In reply to Mike Gilbert from comment #8)
> Another issue that I thought of: When you upgrade OpenSSH, occasionally
> support for a new key format is added. Therefore, we would not want to limit
> key generation to once per boot.

That's fortunately easy to solve, just add 'PartOf=sshd.service sshd.socket'. The following is currently being considered for Fedora. Any feedback welcome.

sshd.service:

[Unit]
Description=OpenSSH server daemon
After=network.target sshd-keygen.service
Wants=sshd-keygen.service

[Service]
EnvironmentFile=/etc/sysconfig/sshd
ExecStart=/usr/sbin/sshd -D $OPTIONS
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
RestartSec=42s

[Install]
WantedBy=multi-user.target

sshd@.service:

[Unit]
Description=OpenSSH per-connection server daemon
Wants=sshd-keygen.service
After=sshd-keygen.service

[Service]
EnvironmentFile=-/etc/sysconfig/sshd
ExecStart=-/usr/sbin/sshd -i $OPTIONS
StandardInput=socket

sshd-keygen.service:

[Unit]
Description=OpenSSH Server Key Generation
ConditionPathExists=|!/etc/ssh/ssh_host_rsa_key
ConditionPathExists=|!/etc/ssh/ssh_host_ecdsa_key
ConditionPathExists=|!/etc/ssh/ssh_host_ed25519_key
PartOf=sshd.service sshd.socket

[Service]
ExecStart=/usr/sbin/sshd-keygen
Type=oneshot
RemainAfterExit=yes

Note that Fedora decided to generate the keys on the first client's when sshd.socket is in use, not when the socket is started.

(In reply to Mike Gilbert from comment #8)
> Given that, I think we should probably just add
> ExecStartPre=/usr/bin/ssh-keygen -A to sshd@.service. The overhead of
> running that command when the keys already exist is trivial.

I would also regrd the conditions as optional. It provides a nice optimization but it's not entirely necessary.

(In reply to Markos Chandras from comment #9)
> arch linux does the following[1]
> 
> sshd@.service
> [Unit]
> Description=OpenSSH Per-Connection Daemon
> After=sshdgenkeys.service
> 
> sshd.socket
> [Unit]
> Conflicts=sshd.service
> Wants=sshdgenkeys.service

Looks correct but does it provide any real advantage to having the After either directly in sshd.socket (when you want your keys ready at boot time) or having Wants in the sshd@.service (when you want your keys generated at first requiest)?

In this case they are generated at boot time without delaying sshd.socket which is a trivial operation and noone in the system should depend on sshd.socket, should they?

I guess Fedora prefers generating at first request time exactly to provide the socket without performing any actions.

> [1] https://www.archlinux.org/packages/core/i686/openssh/download/
> 
> Can we do something similar?

Apart from that one detail it seems Archlinux is using basically the same we concluded for Fedora. If we can actually get a cross-distribution configuration (minus some minor tweaks), it might be good to post it upstream.

I'm trying to achieve a similar thing with NetworkManager which is a bit more complicated as it is expected to *provide* the network-online.service.

https://bugzilla.gnome.org/show_bug.cgi?id=728965
Comment 11 Markos Chandras (RETIRED) gentoo-dev 2014-05-09 20:01:54 UTC
I don't mind if we do what Fedora or Arch do. They are both systemd-only distros so either solution should work fine for Gentoo.

Agreed?

Could anyone attach the diffs for all the existing .service sshd files in portage so I can review them properly?
Comment 12 Mike Gilbert gentoo-dev 2014-05-09 21:47:16 UTC
I guess I don't really understand the point of those ConditionPathExists entries. That is something we would have to update manually every time OpenSSH add a new key.

ssh-keygen -A already checks to see if the necessary keys exist.
Comment 13 Mike Gilbert gentoo-dev 2014-05-09 21:52:36 UTC
Oh, I see Pavel agrees with me there. ^_^
Comment 14 Michael Jones 2022-01-17 20:02:40 UTC
This effected me recently.

Has a solution been agreed upon since the last activity in 2014?