«
»
 

Manage AWS EC2 Snapshots

| November 23rd, 2017

Create and delete snapshots

Create and delete snapshots according to retention plan.

  1. IAM account:
    •  In the AWS Management Console, go to IAM service.  On the right side click on Users then Add User:
    •  Type In user name and select Programmatic access. Then Next:Permissions
    •  On the next page select “Attach existing policy directly” then click on Create Policy
    •  Click on the JSON tab then copy/paste the code below(we will use one policy for creating and deleting snapshots): 
      {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1426256275000",
            "Effect": "Allow",
            "Action": [
                "ec2:CreateSnapshot",
                "ec2:CreateTags",
                "ec2:DeleteSnapshot",
                "ec2:DescribeSnapshots",
                "ec2:DescribeVolumes",
        "ec2:DescribeInstances"				
            ],
            "Resource": [
                "*"
            ]
        },{
      "Sid": "Stmt1422916495000",
      "Effect": "Allow",
      "Action": [
        "ec2:DeleteSnapshot",
        "ec2:DescribeSnapshots"
      ],
      "Resource": [
        "*"
      ]
    }
    ]
}

       

    •  Click on Review policy
    •  Give it a Name and Description, such as CMD-SNAPSHOTS-POLICY. Then Click on Create policy
    •  Back to the Permissions Page. Click on the Refresh button and type in the filter the name of the policy you’ve just created.  Should see the new policy. Select it, then click on Next: Preview and on the next page Create User
    •  From the next page, make a copy of the Access key ID & Secret access key
    • On the server, create a new file, and save it to $HOME/.awssecret, with the following format: 
      Access key id
Secret access key

       

  2. Using ec2-consistent-snapshot from https://github.com/alestic/ec2-consistent-snapshot to take snapshots.On a Centos 7 system, Enable epel: 
    wget dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-11.noarch.rpm
rpm -ivh epel-release-7-10.noarch.rpm

    Then install the required Modules:

    yum install \
  perl-DBI \
  perl-DBD-M ySQL \
  perl-Getopt-Long \
  perl-Pod-Usage \
  perl-File-Slurp \
  perl-IO-Socket-SSL \
  perl-Net-Amazon-EC2 \
  perl-Time-HiRes \
  perl-Net-SSLeay \
  perl-Params-Validate \
  ca-certificates \
  perl-DateTime

    If you are using cPanel, install the required Perl Modules by login to WHM, then Software -> Install Perl Modules. Then install the following:

    DBD::mysql
    Getopt::Long
    Pod::Usage
    File::Basename
    File::Slurp
    IO::Dir
    IO::Socket::SSL
    LWP::UserAgent
    Time::HiRes
    Net::Amazon::EC2
    DateTime::Locale
    DateTime::TimeZone

    Not 100% sure if this is all the needed module, you can see what’s missing when you run the script. Just make a note of the error and install the missing module.

    After that download the script from : https://github.com/alestic/ec2-consistent-snapshot

    Save it to a folder on your system, I choose /aws
    Make it executable: chmod +x /aws/ec2-consistent-snapshot

  3. To delete old  snapshots, I’ve used: ec2-expire-snapshots from https://github.com/alestic/ec2-expire-snapshots
    Needed Perl Modules

    yum install perl-Date-Manip \
 perl-DateTime \
 perl-DateTime-TimeZone \
 perl-DateTime-Format-ISO8601 \
 perl-File-Slurp \
 perl-Time-HiRes \
 perl-Params-Validate \
 ca-certificates \
 perl-DBD-MySQL \
 perl-Net-Amazon-EC2

    If there is a problem with the package perl-Net-Amazon-EC2. Then use cpan (https://goo.gl/fdEPvU):

    yum install -y expat-devel
cpan App::cpanminus  # a better package manager for CPAN
/usr/local/bin/cpanm Net::Amazon::EC2

    Cpanel (Bold = installed in previous step):

    Date::Manip
    DateTime
    DateTime::TimeZone
    DateTime::Format::ISO8601
    File::Slurp
    Time::HiRes
    Params::Validate
    Net::Amazon::EC2
    Moose::Exception::ValidationFailedForInlineTypeConstraint

    After that download the script from: https://github.com/alestic/ec2-expire-snapshots
    Save it to the same folder on your system:
    Make it executable: chmod +x /aws/ec2-expire-snapshots

  4. Create a script and cron job to run the previous scripts:
    create a file called: aws_snapshots

    volume='vol-ID'

echo "AWS Snaphot backup started"
echo "Date: `date`"
echo ""
/aws/ec2-consistent-snapshot --mysql --debug --region eu-west-1 $volume
echo "AWS Snaphot backup Ended"
echo ""
echo "Removing old snapshots"
/aws/ec2-expire-snapshots \
  --region eu-west-1 \
  --keep-most-recent 1 \
  --keep-first-daily 7 \
  --keep-first-weekly 4 \
  --keep-first-monthly 6 \
  $volume 
echo "done"

    (Better explore the options offered by ec2-consistent-snapshot & ec2-expire-snapshots, and change the script accordingly)

    Make the script runnable: chmod +x /aws/aws_snapshots

    add it to your cron job to run daily

Posted on Thursday, November 23rd, 2017 at 4:04 pm in Amazon EC2, Centos, Linux | rss feed for comments| Leave a response, or trackback