AWS Security — Dealing with Exposed Access keys
Exposed AWS Access keys are one of the very common use-cases that you will find across many organizations.
There are certain steps that AWS recommends in such a scenario
i) Determine the access associated with those keys
ii) Invalidating the credentials
iii) Invalidating any temporary credentials issued with the exposed keys
iv) Restore the access with new credentials
v) Review your AWS account
1. Determine the access associated with those keys
Depending on the permissions associated with those exposed keys, it will help you choose the steps that need to be taken and in which order.
i) Exposed keys have read and write access to very sensitive data.
ii) Exposed keys have read access to a public S3 bucket.
Actual Scenario
Consider you are dealing with a production account where the access key is exposed. Directly disabling the key or deleting the key is not the best option. Assume that the key is having only some read access to S3, In this case, if we disable the access key the production application won’t work until we start the application with a newly updated access key. So decide to disable based on the access level the key has.
2. Invalidating the credentials
When we disable or delete the credentials, any application using them might be affected.
Ideally, disabling credentials is recommended instead of deleting because disabled credentials can be enabled back instead of any major production outages.
Actual Scenario
Assume you are working in a production application and the access key is exposed. You are deleting the key instead of disabling it. Once keys are deleted, the Business team is asking to start the servers because they don’t want to have an outage as of now and the impact will be high if we have a production outage now.
Since you deleted the keys there is no way to start again easily. We need to create new keys and update all servers. It will take more time to start all servers and not easy as enabling the keys again.
3. Invalidating any temporary credentials that might have been issued with exposed keys
Temporary credentials can be generated from the access/secret keys. These credentials can have a lifetime from 15 minutes to 36 hours. Ideally, disabling credentials is recommended instead of deleting because disabled credentials can be enabled back instead of any major production outages.
Actual Scenario
Assume you are working on the team where the access key is exposed. Immediately you are disabling or deleting the access key. Do you know whether temporary credentials work after disabling or deleting access keys? Obviously, if you are deleting the IAM user itself then it won't work but it will impact current applications.
Yes, temporary credentials will work till their lifetime. To restrict access to these temp keys we need to add an explicit deny using a proper policy like below.
{ “Version”: “2012–10–17”,
“Statement”: [{
“Effect”: “Deny”,
“Action”: “*”,
“Resource”: “*”,
}]
}
This is an interesting one and I assume most of our understanding is different from this use case.
4. Restore the access with new credentials
Because access keys were disabled / deleted in previous steps, consider the following options to restore the access:
i) Consider creating a new pair of access/secret keys.
ii) Instead of using long-term keys, consider using IAM roles or federation.
5. Review your AWS account
Review all the available S3 buckets as well as CloudTrail logs to see what actions might have been performed on your AWS resources.
You might want to restore data from a backup that was made before the credentials were exposed.
We also need to delete unwanted resources created using exposed access keys.
Best Practices for Managing AWS Keys
1. Don’t create an access key for the root user
Access Key associated with the root user will have access to all the resources including billing information. We can’t restrict these permissions.
2. Don’t embed access keys directly into code
The AWS SDKs and the AWS Command Line Tools allow you to put access keys in known locations so that you do not have to keep them in code.
The most secure method is to load the credentials from AWS Identity and Access Management (IAM) roles for Amazon EC2.
3. Rotate access keys periodically
Regularly rotating your IAM credentials helps prevent a compromised set of IAM access keys from accessing components in your AWS account.
4. Remove unused access keys
Like whenever an employee leaves the company we need to remove the keys associated with that person.
5. Configure multi-factor authentication for your most sensitive operations
Assume we are running a production application in an EC2 instance and we don’t want the user to shut it down. In this case, we can map the MFA with that particular API operation so that users who logged in without MFA can’t perform that action on the mentioned services.
6. Use IAM roles instead of long-term access keys
The problem with Long-term access keys is they never expire until we manually revoke them. So it’s better to use temporary credentials obtained through IAM Roles. The token expires after a short period of time. The risk is very minimal in this case even though accidentally our temporary access key was exposed.
7. Access the mobile app using AWS access keys
As part of the incident response plan, we can have a mobile app to access our AWS account. This app will have access to some of the services and their features.
