Data Encryption at Rest: Preparing for EBS Volumes Remediation

Taking the First Steps Toward Secure Data at Rest

In the previous post , I discussed the importance of encrypting data at rest and introduced the powerful combination of AWS tools and the Prowler open-source framework. After running Prowler’s security assessment, you may have identified some unencrypted resources, particularly EBS volumes attached to EC2 instances.

Remediating unencrypted EBS volumes is critical for ensuring that sensitive data is protected, but it requires a careful, planned approach.

Before diving into remediation, the first step is to collect detailed information on the unencrypted volumes and the EC2 instances they are attached to.

Collecting Detailed EC2 and EBS Information

To prepare for remediation, I’ve created a Python script (ebs-vols-report.py) that generates a report of all your EC2 instances, the EBS volumes attached to them, and their encryption status. Running this script gives you a clear snapshot of which EBS volumes are unencrypted and provides detailed information that will guide you through the remediation process.

What Does the Script Do?

The script uses the AWS SDK for Python (boto3) to interact with your AWS environment. Here’s a breakdown of how it works:

Get Instance and Volume Information:

• The script connects to your AWS account using the boto3 session with the provided AWS profile and region. It then retrieves all EC2 instances in the region using the ec2_resource object.
• For each EC2 instance, it collects the instance ID, instance name (using the Name tag), and its current state (powered on or off).
• It loops through each instance’s block device mappings to gather the attached EBS volume IDs.
• The script calls the describe_volumes function to get more detailed information about each volume, including its encryption status.

Write the Data to a CSV File:

• The script writes the collected data (instance ID, name, state, volume ID, and encryption status) to a CSV file (ec2_instance_data.csv), providing a structured report of which instances have unencrypted volumes attached.
• This report serves as the baseline for your remediation process. You will use it to identify which instances require encryption updates, and later, you can run the report again to validate that your remediation was successful.

How to Run the Script

python ebs-vols-report.py --region us-west-2 --profile your-aws-profile

The script takes two arguments:

• --region: The AWS region where your instances are running (e.g., us-west-2).
• --profile: The AWS profile you’ve set up locally to authenticate and connect to your AWS account.

The output will look something like this:

Once you have this detailed information in hand, you can move on to the actual remediation process.

The Full Python Script

import boto3
import csv
import argparse

# Function to get the instance's 'Name' tag
def get_instance_name(instance):
    for tag in instance.tags or []:
        if tag['Key'] == 'Name':
            return tag['Value']
    return 'N/A'  # If no 'Name' tag is found, return 'N/A'

# Function to get instance state and volume encryption status
def get_instance_data(ec2_resource, ec2_client):
    instance_data = []

    # Loop through all EC2 instances
    for instance in ec2_resource.instances.all():
        instance_id = instance.instance_id
        instance_name = get_instance_name(instance)
        state = instance.state['Name']  # powered on/off state

        # Get the block device mappings (EBS volumes attached to the instance)
        for device in instance.block_device_mappings:
            if 'Ebs' in device:
                volume_id = device['Ebs']['VolumeId']

                # Get the volume details, including the encryption status
                volume = ec2_client.describe_volumes(VolumeIds=[volume_id])['Volumes'][0]
                encrypted = volume['Encrypted']

                instance_data.append({
                    'Instance ID': instance_id,
                    'Instance Name': instance_name,
                    'State': state,
                    'Volume ID': volume_id,
                    'Encrypted': encrypted
                })

    return instance_data

# Function to write the instance data to a CSV file
def write_to_csv(data, filename='ec2_instance_data.csv'):
    with open(filename, mode='w', newline='') as file:
        writer = csv.DictWriter(file, fieldnames=['Instance ID', 'Instance Name', 'State', 'Volume ID', 'Encrypted'])
        writer.writeheader()
        writer.writerows(data)

    print(f"Data written to {filename}")

def main():
    # Create an argument parser for region and profile
    parser = argparse.ArgumentParser(description="Script to list EC2 instances and EBS volumes with encryption status")
    parser.add_argument('--region', required=True, help="The AWS region to use")
    parser.add_argument('--profile', required=True, help="The AWS profile to use")

    args = parser.parse_args()

    # Create a session using the provided profile and region
    session = boto3.Session(profile_name=args.profile, region_name=args.region)

    # Create an EC2 resource and EC2 client
    ec2_resource = session.resource('ec2')
    ec2_client = session.client('ec2')

    # Get instance data and write to CSV
    instance_data = get_instance_data(ec2_resource, ec2_client)
    write_to_csv(instance_data)

if __name__ == "__main__":
    main()

Beginning the Remediation Process

Now that you know which EC2 instances have unencrypted EBS volumes, it’s time to make sure you fully understand the remediation process for getting them encrypted. Here’s the step-by-step process for securing the unencrypted volumes:

Determine KMS Key to Use: It is very important to have your destination in mind. For example, if your goal is simplicity, you may use the built-in AWS KMS provided default key for encrypting your volumes. However, if you are going to be using something like the AWS Backup service to copy recovery points containing these volumes across AWS accounts or AWS regions, you will need to pivot your KMS key strategy accordingly. I highly recommend you create a new multi-region KMS key for this exclusive purpose.

Power Down the EC2 Instance: For safety and to ensure data integrity, stop the EC2 instance. This prevents any changes to the volume during the encryption process. It is super important to understand that there is no way to perform a live re-encryption operation in place. You must plan for downtime accordingly. The amount of downtime for each instance depends on a number of factors, such as number of EBS volumes attached to the instance, the EBS volume type, EBS volume size, and how much of the volume is in active use (in terms of blocks written).

Detach the EBS Volumes: Using the AWS Management Console or AWS CLI, detach the unencrypted EBS volumes from the instance.

Create an Unencrypted Snapshot: Before making any modifications, create a snapshot of the unencrypted volume. This acts as a backup and ensures that no data is lost during the process.

Copy the Snapshot with Encryption: Create a new encrypted snapshot from the unencrypted one. This step uses your KMS key to encrypt the data.

Create a New EBS Volume: Once you have the encrypted snapshot, create a new EBS volume from it.

Reattach the Encrypted EBS Volume: Attach the newly encrypted EBS volume to your EC2 instance.

Power the EC2 Instance Back On: Finally, restart the instance, and it will now have encrypted EBS volumes attached.

Validating the Remediation

After completing the remediation process, you can run the ebs-vols-report.py script again. This will allow you to verify that all previously unencrypted EBS volumes are now encrypted. The new report should show True under the “Encrypted” column for all the volumes you remediated.

Next Steps: Automating the Remediation

Remediation is a crucial part of securing your AWS environment, and the preparation phase is just as important. By using this Python script, you’ve taken the first step in gathering the data needed to begin remediation. In the next post, we’ll cover a Python script to fully automate the encryption process, streamlining the work even further!

Stay tuned as we continue to secure your AWS resources, one encrypted volume at a time!