Developer.ps

Create and delete snapshots according to retention plan.

1. IAM account:
   •  In the AWS Management Console, go to IAM service.  On the right side click on Users then Add User:
     

     

     AWS IAM User

     
     
   •  Type In user name and select Programmatic access. Then Next:Permissions
   •  On the next page select “Attach existing policy directly” then click on Create Policy
   •  Click on the JSON tab then copy/paste the code below(we will use one policy for creating and deleting snapshots):

     {
         "Version": "2012-10-17",
         "Statement": [
             {
                 "Sid": "Stmt1426256275000",
                 "Effect": "Allow",
                 "Action": [
                     "ec2:CreateSnapshot",
                     "ec2:CreateTags",
                     "ec2:DeleteSnapshot",
                     "ec2:DescribeSnapshots",
                     "ec2:DescribeVolumes",
             "ec2:DescribeInstances"				
                 ],
                 "Resource": [
                     "*"
                 ]
             },{
           "Sid": "Stmt1422916495000",
           "Effect": "Allow",
           "Action": [
             "ec2:DeleteSnapshot",
             "ec2:DescribeSnapshots"
           ],
           "Resource": [
             "*"
           ]
         }
         ]
     }

      

   •  Click on Review policy
   •  Give it a Name and Description, such as CMD-SNAPSHOTS-POLICY. Then Click on Create policy
   •  Back to the Permissions Page. Click on the Refresh button and type in the filter the name of the policy you’ve just created.  Should see the new policy. Select it, then click on Next: Preview and on the next page Create User
     

     

     
     
   •  From the next page, make a copy of the Access key ID & Secret access key
   • On the server, create a new file, and save it to $HOME/.awssecret, with the following format:

     Access key id
     Secret access key

      

2. Using ec2-consistent-snapshot from https://github.com/alestic/ec2-consistent-snapshot to take snapshots.On a Centos 7 system, Enable epel:

   wget dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-11.noarch.rpm
   rpm -ivh epel-release-7-10.noarch.rpm

   Then install the required Modules:

   yum install \
     perl-DBI \
     perl-DBD-M ySQL \
     perl-Getopt-Long \
     perl-Pod-Usage \
     perl-File-Slurp \
     perl-IO-Socket-SSL \
     perl-Net-Amazon-EC2 \
     perl-Time-HiRes \
     perl-Net-SSLeay \
     perl-Params-Validate \
     ca-certificates \
     perl-DateTime
   

   If you are using cPanel, install the required Perl Modules by login to WHM, then Software -> Install Perl Modules. Then install the following:

   DBD::mysql
   Getopt::Long
   Pod::Usage
   File::Basename
   File::Slurp
   IO::Dir
   IO::Socket::SSL
   LWP::UserAgent
   Time::HiRes
   Net::Amazon::EC2
   DateTime::Locale
   DateTime::TimeZone

   Not 100% sure if this is all the needed module, you can see what’s missing when you run the script. Just make a note of the error and install the missing module.

   After that download the script from : https://github.com/alestic/ec2-consistent-snapshot

   Save it to a folder on your system, I choose /aws
   Make it executable: chmod +x /aws/ec2-consistent-snapshot

3. To delete old  snapshots, I’ve used: ec2-expire-snapshots from https://github.com/alestic/ec2-expire-snapshots
   Needed Perl Modules

   yum install perl-Date-Manip \
    perl-DateTime \
    perl-DateTime-TimeZone \
    perl-DateTime-Format-ISO8601 \
    perl-File-Slurp \
    perl-Time-HiRes \
    perl-Params-Validate \
    ca-certificates \
    perl-DBD-MySQL \
    perl-Net-Amazon-EC2

   If there is a problem with the package perl-Net-Amazon-EC2. Then use cpan (https://goo.gl/fdEPvU):

   yum install -y expat-devel
   cpan App::cpanminus  # a better package manager for CPAN
   /usr/local/bin/cpanm Net::Amazon::EC2

   Cpanel (Bold = installed in previous step):

   Date::Manip
   DateTime
   DateTime::TimeZone
   DateTime::Format::ISO8601
   File::Slurp
   Time::HiRes
   Params::Validate
   Net::Amazon::EC2
   Moose::Exception::ValidationFailedForInlineTypeConstraint

   After that download the script from: https://github.com/alestic/ec2-expire-snapshots
   Save it to the same folder on your system:
   Make it executable: chmod +x /aws/ec2-expire-snapshots

4. Create a script and cron job to run the previous scripts:
   create a file called: aws_snapshots

   volume='vol-ID'
   
   echo "AWS Snaphot backup started"
   echo "Date: `date`"
   echo ""
   /aws/ec2-consistent-snapshot --mysql --debug --region eu-west-1 $volume
   echo "AWS Snaphot backup Ended"
   echo ""
   echo "Removing old snapshots"
   /aws/ec2-expire-snapshots \
     --region eu-west-1 \
     --keep-most-recent 1 \
     --keep-first-daily 7 \
     --keep-first-weekly 4 \
     --keep-first-monthly 6 \
     $volume 
   echo "done"

   (Better explore the options offered by ec2-consistent-snapshot & ec2-expire-snapshots, and change the script accordingly)

   Make the script runnable: chmod +x /aws/aws_snapshots

   add it to your cron job to run daily

Posted on Thursday, November 23rd, 2017 at 4:04 pm in Amazon EC2, Centos, Linux | rss feed for comments| Leave a response, or trackback