Configuring SSH for Public Key Authentication
In part one of this series, we showed you how to configure a Windows 2019 EC2 instance to run an OpenSSH Server to pave the way for secure, remote development. This post walks through the setup of pre-shared keys for password-less login. Part three covers installing and configured VS Code’s remote development capabilities.
Setting up Private and Public Keys on the Local Machine
On your local developer machine, you need to generate a public and private key. Linux and Mac users will have all the tools you need. Windows users, you’ll need to install the ssh client on your local development machine. The remote development server needs to have the line below in its sshd_config and to be restarted like we did in Part 1.
PubkeyAuthentication yes
On the local development machine, we need to make sure we have a client.
In an elevated Powershell, run:
# Make sure that OpenSSH is available for your Windows
Get-WindowsCapability -Online | ? Name -like 'OpenSSH*'
# Install the OpenSSH Client
Add-WindowsCapability -Online -Name OpenSSH.Client~~~~0.0.1.0
Open up your favorite command line:
ssh-keygen
We’re going to go through the defaults without a password using RSA. Best practices recommend generating a key with a password. You’ll be prompted to provide the password whenever you try to use the key unless you load it into an agent as detailed here.
Once you have your key loaded, copy the contents of the id-rsa.pub and get ready to install it on the server. It will be located in the user’s ~/.ssh/id_rsa.pub file.
The last step is to load the ssh key into the local development machine’s key agent. The steps are different for Windows, Mac and Linux. Github provides an exhaustive guide here.
Installing the Public Key
For WSL Ubuntu from a Linux or Mac, you can install the key onto the OpenSSH Server with one command. You will need the destination user created and configured on the development server. You will need to password-based authentication enabled to use this command, because you must login as that user. Windows users, unfortunately, are out of luck on both sides. There’s no ssh-copy-id for Windows local machines and the OpenSSH Server for Windows lacks the Linux utilities that copy the key once it gets to the server.
For the WSL Ubuntu, you can run:
ssh-copy-id demo@{your ec2 dns name or ip address}
When prompted, provide the password we created in part 1.
You should see output like:
demo@ec2-3-101-29-193.us-west-1.compute.amazonaws.com's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'demo@ec2-3-101-29-193.us-west-1.compute.amazonaws.com'"and check to make sure that only the key(s) you wanted were added.
What this utility is doing is SCPing the specified public key to the users /home/demo/.ssh/authorized_keys file and setting that appropriate permission. If you’re running a windows client or have a windows server, you won’t have the convenience of ssh-copy-id.
You can however either WSL or SSH into the WSL Ubuntu shell and do everything there. This is all done from the user’s perspective, so the files need to be created in their home directory and they need to own it.
#Create the directory if it doesn’t exist
mkdir -p /home/demo/.ssh
touch /home/demo/.ssh/authorized_keys
sudo chown -R demo:dmemo /home/user/.ssh
Now, copy the contents of your id_rsa.pub file into /home/demo/.ssh/authorized_keys
And you’re done.
Installing the Key by Hand (Windows Administrator)
By default, OpenSSH Server for Windows comes with a special configuration for Windows Administrators. This is where things get kind of confusing.
If your user is an administrator, you need to create a special authorized key file.
%PROGRAMDATA%\ssh\administrators_authorized_keys
Log into RDP, create and edit that file. Then save the contents of your client machine’s id_rsa.pub key to that file for administrators.
We’re not quite done yet. The Microsoft OpenSSH team has been very helpful with answers. They are going to fix a permissions gotcha that exists at the time of this writing. If you install the key and can’t connect, it’s probably a permissions issue.
Be sure:
- The user on the development server is an administrator
- You have PubKeyAuthentication set to yes in the sshd_conf
- That the permissions are correct on the authorized_administrator_keys.
To fix the file permissions
Open up Windows Explorer
Go to %PROGRAMDATA%\ssh
Open up the Properties dialog for administrators_authorized_keys
Under Security > Advanced
Disable inheritance.
Remove any Permission Entry other than the Administrators group and SYSTEM.
If you’re looking for the OpenSSH Utils powershell cmdlets to fix this problem, don’t. Those have been de-listed from the nuget package repository and powershell gallery in favor of the OpenSSH Server handling the permissions themselves.
Installing the Key by Hand (Windows Regular User)
Regular users use the standard .ssh file in their user directory at HOMEPATH\.ssh\authorized_keys
Log into RDP, create and edit that file. Then save the contents of your client machine’s id_rsa.pub key to that file for administrators.
Troubleshooting Windows OpenSSH Server
If you’re reading this, welcome to the “SSH Server Administrator Bootcamp.” SSH is rock solid, but it likes a very specific kind of setup and tends to silently deny you access.
If the keys continue to fail, it’s time to get some diagnostic logging. You can turn on the SysLogFacility as detailed here. There’s an easier and more temporary way too.
Stop your OpenSSH server in services.
Open a terminal and go to %PROGRAMDATA%\ssh
Here, you can run the OpenSSH Server from the command line and see all of its output with sshd -ddd
The -d flag specifies levels of diagnostic output. You need three to get to “debug3”.
There will be a lot of chatter as it tries to authenticate.
If you see:
debug3: Bad permissions. Try removing permissions for user: S-1-5-11 on file C:/ProgramData/ssh/administrators_authorized_keys.
Authentication refused.
That’s SSH complaining about permissions on that file. It’s very strict about who can access it.
Once you have the OpenSSH Server accepting your key, kill the command line and restart the service. There’s many ways your key can fail. Some common pitfalls are permissions, character encoding and trying to install an administrator key in their regular .ssh authorized keys file. Fortunately, it’s a very chatty log file once you get used to its unique brand of diagnostic gibberish.
Another good technique is the -v “verbose” flag on the ssh client itself. On the local development machine, you can log verbose connection information with:
ssh -vvv {your ec2 dns name or ip address}
Turn off Password Based Authentication
SSH best practice recommends turning of password-based authentication. It’s a balancing act based on your security needs. If you have your SSH server configured to only allow certain connections through AWS’ Inbound Security rules, you won’t really need to worry about brute force attacks and the ease of ssh-copy-id will make it easy for you to upload keys if you know that user’s username and password.
If you do want to turn them off, then edit your sshd_config on the OpenSSH Server and restart it.
Agent Forwarding for WSL Ubuntu
One major advantage of running OpenSSH Server through WSL Ubuntu is that you get key-agent forwarding. By enabling it on the server, loading it into the developer’s local machine key agent and registering the public key with the repository, you will have one private / public key to access all the resources.
Setting up another key for Windows Development Servers
Until OpenSSH Server for Windows supports key agent forwarding, you are going to have to manage two sets of SSH keys. The first key we setup will handle the connection from the developer’s local machine to the development server. If you create a second set of keys on the development server and install that second public key on the repository, you will be able to have password-less authentication on the server.
Adding the Public Key to your Repository
With the public key of the client configured, it’s time to add it to the repository. We’re using Bitbucket in this example, but the process is the same for Github or Gitlab. Copy the public key for your user and paste it into the repository settings. Then, the user will be able to pull and push without ever having to worry about user keys.
Important: Be sure to clone your repository using the ssh url and not the user-specific https one.
You can confirm your remote url with:
git remote -vAnd change it to the ssh version with:
git remote set-url --push origin git@github.com/demo/master
You will want to make sure you are using the correct url for logging into your repository. If you can clone and pull without providing a username and password, you can move on to part three and have a repository-aware remote development environment.
Brian Hatchet is a Senior BI Consultant at DesignMind.