Request a Quote

Lambda function to monitor your EC2 snapshots

Lambda function to monitor your EC2 snapshots

August 30, 2018 at 1:05 pm By Arjun R Dev
A snapshot is a point-in-time copy of data. The best thing about snapshot over a normal backup is that effortlessness in rollback. Those who use AWS ec2 will be taking snapshots to be on the safer side. Most of us will be using some automated scripts to create snapshots since manually taking a snapshot is a time-consuming process. Most of such scripts will be also having a retention period set so that old snapshots will be removed. But there are situations in which something could go wrong and the retention may fail and we might not be aware of it since we have multiple instances in different regions. And that can lead us to consume so much space and paying a huge amount for that. That is why we build a lambda script which can monitor all the snapshots that have been taken from a volume which is associated to an ec2 instance and search if there is any snapshot which is older than the retention date specified in the lambda function.  

Prerequisites:

  • Ec2 instances
  • Snapshots with the tag “backup”For the lambda function to work it should have access to the ec2 instances and write CloudWatch logs. We will have to create a new IAM role and attach a policy which will allow the lambda to interact with the ec2. You can give any convenient name for the role. You can make use of the policy which I have created or you can create your own.
{
	"Version": "2012-10-17",
	"Statement": [
    	{
        	"Effect": "Allow",
        	"Action": [
            	"logs:*"
        	],
        	"Resource": "arn:aws:logs:*:*:*"
    	},
    	{
        	"Effect": "Allow",
        	"Action": "ec2:Describe*",
        	"Resource": "*"
    	}
	]
}

Steps to create IAM role and to attach policy is as below :

  • Go to Services, IAM, Create a new Role
  • Select the option Lambda
  • Don’t select any policy, click Next, and Create Role.
  • Enter the role name (Eg:ebs-lambda-worker)
  • Select the new role, and click attach policies
  • Click on the option Create Policy
  • Select the option json and insert the content of the above snippet.
 
  • Click the button Review Policy button.
  • Provide a name for the policy and click on the button create policy.
  • Now select the policy that you have created and click on the button policy actions and select the option attach.
Now we will have to create a lambda function. You can follow the below steps to create a lambda function. Steps to create the Lambda function:
  • Go to Services, Lambda, and click Create a Lambda Function
  • Write a name for it
  • Select Python 2.7 as a Runtime option
  • Select the previously created IAM role
  • Click Create Function
  • Paste the code below in the inline editor.
import boto3
from botocore.exceptions import ClientError
from datetime import datetime,timedelta

def lambda_handler(event, context):
    

	filters = [{'Name': 'tag-key', 'Values': ['backup']}]

    
	#define retention period(in days)
	retention_days = 10
	now = datetime.now()
    
	#create EC2 client
	ec2 = boto3.client('ec2')

    
	#list of regions
	regions = ec2.describe_regions().get('Regions',[] )
	numberofsnapshot = 0
	old = 0
	# search in regions for instances
	for region in regions:
    	print "Checking region %s " % region['RegionName']
    	reg=region['RegionName']
    	ec2 = boto3.client('ec2', region_name=reg)
    	result = ec2.describe_volumes( Filters=[{'Name': 'status', 'Values': ['in-use']}])
    	for volume in result['Volumes']:#get the volume ID of the instance
        	result1 = ec2.describe_snapshots(Filters=filters)
        	for snapshot in result1['Snapshots']:#get the snapshot details and store it in to the variable snapshot
            	print "Checking snapshot %s which was created on %s" % (snapshot['SnapshotId'],snapshot['StartTime'])
            	numberofsnapshot = numberofsnapshot + 1
            	time = snapshot['StartTime'].replace(tzinfo=None)
            	if (now - time) > timedelta(retention_days):#check if the timedelta is greater than retention days
                	old = old + 1
        	print "for volume %s found %s snapshots number of snapshots older than the retention date is %s  "% (volume['VolumeId'],numberofsnapshot,old)
  • Make sure that timeout value is more than 1 minute.
  • Please be noted that this lambda function will check for the snapshots which is having a tag “backup”. Hence if you are using any scripts to generate snapshots please do make sure that the snapshot is having a tag as “backup”.
  • Now click on the Test and it will execute the function.
  • Once the execution is completed you will be able to see the result just below the lambda function.
 

Conclusion

Using this lambda function you will be able to monitor the snapshots that you have taken using scripts and make sure that there are no older snapshot present in your account. This Lambda function will help you save lot of your time and effort in verifying old snapshots are not consuming the space and your money.