Capturing EC2 Private Keypair for Automation or CI/CD via AWS CDK
Have you ever had a situation where you needed access to a set of EC2 instances via SSH, but you didn’t have access as the original owner had moved on, and the associate private key wasn’t available (as private keys are not accessible after initial setup)? Or perhaps you wanted to run automation against a set of EC2 instances from within a private only network access using a tool like Ansible. Perhaps you were setting up a CI/CD pipeline and communication between the primary and worker nodes needs to be established. This article will walk through a repeatable mechanism for this setup using infrastructure as code.
AWS CDK
Version 2 of Amazon Web Services Cloud Development Kit (CDK) became available last year (2021). The CDK allows developers to use some of the most popular coding languages to develop Cloud Formation Templates; AWS’s infrastructure as code specification. Writing Cloud Formation Templates (CFTs) can be an onerous task, but with the CDK a developer can use API and generate, deploy, and destroy CFTs. The examples shown in this walkthrough are in Python but can be transformed to the supported language of your choice.
Step-by-step
Let’s walk through an example CDK-based project that will:
1. Create a virtual private network (VPC)
2. Generate a keypair for accessing EC2 instances that will be created via an AWS Lambda Function
3. Store the private key within AWS Secrets Manager
4. Create an autoscale group that can be accessed using the generated keypair
5. Create a manager EC2 instance that captures the private key on the system so that it can be used to access the instances within the autoscale group.
These five different items are all captured in the following example GitHub repository:
Creating a VPC
There are many examples of the basic setup of a VPC. We will focus our attention on the net.pyfile. Below you can see the creation of a simple virtual private network.
Create Lambda Custom Resource
Now that we have a network setup we can create the keypair that will be used in the instances the keypair manager will have access to. The code below, also in the net.py file, creates:
- An IAM role that allows the Lambda function to run and store the private key result in AWS Secrets Manager
- A security group
- The lambda function definition
- A custom resource to enable the function to be triggered in the flow of the Cloud Formation deployment
Keypair Generation and Storage
Now let’s look at the actual Lambda function, ec2_keypair_init.pyfile, which will be creating the keypair using the Boto3 API and storing the resulting private key in the AWS Secrets Manager.
Creating Keypair Managed Instances
With the keypair created and the private key stored in AWS Secrets Manager we can proceed to deploy EC2 instances with the keypair. The next section of code, systems.py, provides an example autoscale group that utilizes the keypair created by the Lambda function.
Setup Keypair Manager Instance
Next we will setup the keypair manager instance, manager.py, and take a look at the bootstrap script.
This manager instance can be setup with any keypair for access and it will have access to the private key of the managed instances. The manager can do this as it has an IAM role that allows it to read from the AWS Secrets Manager and it has the secret name for the private key. The secret name is passed to the the bootstrap file which uses the AWS CLI to create the PEM file as can be seen below in the bootstrap script, bootstrap_manager_node.sh.
Synopsis
Hopefully, this story has enlightened you to an approach for keypair management of EC2 instances. We walked through the creation of the Lambda function via CDK and the Boto3 interactions to create and store the keypair. We looked a the manager creation flow and how it obtains the private key via its bootstrap script. Best of all these steps are captured in repeatable infrastructure-as-code.
