SSH Access Infrastructure SOP

Contents

Contact Information

Owner

sysadmin-main

Contact

#fedora-admin or admin@fedoraproject.org

Location

All

Servers

All IAD2 and VPN Fedora machines

Purpose

Access via ssh to Fedora project machines.

Introduction

This SOP contains information on how to setup access to fedoraproject.org servers via ssh from your client machines.

All access requires ssh (secure shell), using keys. See https://docs.fedoraproject.org/en-US/fedora/latest/system-administrators-guide/infrastructure-services/OpenSSH/ for more detailed information on OpenSSH.

Note that this SOP has nothing to do with actually gaining access to specific machines. For that you MUST be in the correct group for shell access to that machine. This SOP simply describes the process once you do have valid and appropriate shell access to a machine.

SSH configuration

  1. Generate a ssh keypair on your local machine if you don’t already have one. run: 'ssh-keygen -t ed25519' Make sure you enter a strong passphrase you can remember or have stored in a password manager. If you are only going to connect to RHEL9+ and Fedora machines, you can also use a ecdsa_sk key with a FIDO2 hardware device.

  2. Setup your local client ssh config

    vi ~/.ssh/config

    (Or use any text editor you are comfortable with)

    This file, and any keys, need to be mode 600, or you will get a "Bad owner or permissions" error. The .ssh directory must be mode 700.

    then, add the following contents to that text file:

    Host bastion.fedoraproject.org
  HostName bastion.fedoraproject.org
  User FAS_USERNAME (all lowercase)
  ProxyCommand none
  ForwardAgent no
  VerifyHostKeyDNS yes
Host *.iad2.fedoraproject.org *.qa.fedoraproject.org 10.3.160.* 10.3.161.* 10.3.163.* 10.3.165.* 10.3.167.* 10.3.171.* *.vpn.fedoraproject.org
  ProxyJump bastion.fedoraproject.org
Host batcave01
  HostName %h.iad2.fedoraproject.org

    Note that there are 2 bastion servers: bastion01.fedoraproject.org and bastion02.fedoraproject.org. The 'bastion.fedoraproject.org' name should match whichever one is primary at any time. If for some reason you get connection refused or unreachable messages from it, you can manually change the above to specifically point to bastion01 or bastion02. Normally this should not be needed.

  3. Setup the fedora infrastructure ssh hostkey certificate authority:

    Download https://admin.fedoraproject.org/ssh_known_hosts and add it to (or create if it does not yet exist) a ~/.ssh/known_hosts file.

    This tells OpenSSH to trust any host keys that are signed with the fedora infrastructure ssh certificate authority. This allows you to know when you login to a machine that it’s not been replaced or tampered with.

PuTTY SSH configuration

You can configure Putty the same way by doing this:

  1. In the session section type batcave01.iad2.fedoraproject.org port 22

  2. In Connection:Data enter your FAS_USERNAME

  3. In Connection:Proxy add the proxy settings

  • ProxyHostname is bastion-iad01.fedoraproject.org

  • Port 22

  • Username FAS_USERNAME

  • Proxy Command plink %user@%proxyhost %host:%port

  1. In Connection:SSH:Auth remember to insert the same key file for authentication you have used on FAS profile

Using OpenSSH

You can use openssh from any terminal to access machines you are granted access to:

'ssh batcave01.iad2.fedoraproject.org'

It’s important to use the fully qualified domain name of the host you are trying to access so that the certificate matches correctly. Otherwise you may get a notice that the ssh host key is unknown.

SSH Agent forwarding

You should normally have:

ForwardAgent no

For Fedora hosts (this is the default in OpenSSH). You can override this on a per-session basis by using '-A' with ssh. SSH agents could be misused if you connect to a compromised host with forwarding on (the attacker can use your agent to authenticate them to anything you have access to as long as you are logged in). Additionally, if you do need SSH agent forwarding (say for copying files between machines), you should remember to logout as soon as you are done to not leave your agent exposed.

Troubleshooting

  • 'channel 0: open failed: administratively prohibited: open failed'

    If you receive this message for a machine proxied through bastion, then bastion was unable to connect to the host. This most likely means that tried to SSH to a nonexistent machine. You can debug this by trying to connect to that machine from bastion.

  • if your local username is different from the one registered in FAS, please remember to set up a User variable (like above) where you specify your FAS username. If that’s missing SSH will try to login by using your local username, thus it will fail.

  • ssh -vv is very handy for debugging what sections are matching and what are not.

  • If you get access denied several times in a row, please consult with #fedora-admin.

  • If you are running an OpenSSH version less than 5.4, then the -W option is not available. In that case, use the following ProxyCommand line instead:

    ProxyCommand ssh -q bastion.fedoraproject.org exec nc %h %p

How does ssh ProxyJump work?

The ProxyJump command creates a normal ssh connection using your fas username to connect to bastion.fedoraproject.org. Then, it forwards a port over this connection from your client directly to the remote host. This second connection is a fully encrypted ssh connection, meaning the bastion host cannot see anything going over it.

A connection is established to the bastion host:

+-------+            +--------------+
|  you  | ---ssh---> | bastion host |
+-------+            +--------------+

Your client then connects through the Bastion and reaches the target server:

+-----+                  +--------------+                +--------+
| you |                  | bastion host |                | server |
|     | ===ssh=over=bastion============================> |        |
+-----+                  +--------------+                +--------+

Want to help? Learn how to contribute to Fedora Docs ›