Bootstrap FreeKB - Amazon Web Services (AWS) - Auto Scale EC2 Instances using CloudWatch
Amazon Web Services (AWS) - Auto Scale EC2 Instances using CloudWatch


This assumes you have already configured the aws command line tool. If not, check out my article on Getting Started with the AWS CLI.

An Auto Scaling Group let's you automatically start and stop EC2 instances based on some condition, such as creating and starting up EC2 instances when requests load is high or stopping and terminating EC2 instances when requests load is low. This is often used in conjunction with CloudWatch Alarms, where the Alarm checks some condition such as requests load or CPU Utilization and then invokes the Auto Scaling Group to create/startup EC2 instances or to stop/terminate EC2 instances. For example, you may have an Auto Scaling Group that has minimum 1 and maximum 3, meaning that the Auto Scaling Group will have 1 EC2 instance when load is low and scale up to 3 EC2 instances when load is high.

To use Auto Scaling with CloudWatch, the Auto Scale must have a step scaling or simple scaling policy which will scale out (increase capacity) or scale in (decrease capacity) when an alarm threshold is breached.

Cloudwatch alarms can be used to do something, such as:

  • Publish message to a Simple Notifcation Service (SNS) topic
  • Reboot an EC2 instance
  • Auto Scale to create/start additional EC2 instances or to stop/terminate EC2 instances (this article)

When some conditiion is met, such as:

The aws autoscaling create-auto-scaling-group command can be used to create an EC2 Auto Scaling Group.

You may also want to set an instance warm up period. The warm up period is the number of seconds that must elapse after the instance has started up for the instance to count against the number of instances that should be up and running. For example, 300 seconds is a typical default warm up period. 

aws autoscaling create-auto-scaling-group \
    --auto-scaling-group-name my-auto-scaling-group \
    --launch-template LaunchTemplateId=lt-0706d84b6a79fd1d6 \
    --min-size 1 \
    --max-size 3 \
    --default-cooldown 1 \
    --default-instance-warmup 120 \
    --vpc-zone-identifier "subnet-02b9845e7366bdf89,subnet-0f015da3a1e164304,subnet-02b9845e7366bdf89"

 

The aws autoscaling describe-auto-scaling-groups command can be used to list the Amazon Resource Number (ARN) of the Auto Scaling Groups you have created.

~]$ aws autoscaling describe-auto-scaling-groups | grep AutoScalingGroupARN
            "AutoScalingGroupARN": "arn:aws:autoscaling:us-east-1:123456789012:autoScalingGroup:c21a1c8a-3c0a-4410-89e2-015100f3707f:autoScalingGroupName/my-auto-scaling-group",

 

The Auto Scaling Group must have a simple scaling or step scaling policy to be able to be used with CloudWatch Alarms. Let's create a scale out (increase capacity) step scaling policy. In this example:

  • Increase the instance count by 10 percent when the value of the metric is greater than or equal to 60 percent but less than 75 percent
  • Increase the instance count by 20 percent when the value of the metric is greater than or equal to 75 percent but less than 85 percent
  • Increase the instance count by 30 percent when the value of the metric is greater than or equal to 85 percent

In a moment, we will create the CloudWatch alarm. The alarm will be breached when NetworkIn exceeds 1 million bytes (that's 1 MB of network in traffic). So, in this example:

  • Increase the instance count by 10 percent when NetworkIn is greater than or equal to 600,000 bytes but less than 750,000 bytes
  • Increase the instance count by 20 percent when NetworkIn is greater than or equal to 750,000 bytes but less than 850,000 bytes
  • Increase the instance count by 30 percent when NetworkIn is greater than or equal to 850,000 bytes
aws autoscaling put-scaling-policy \
  --auto-scaling-group-name my-auto-scaling-group  \
  --policy-name my-step-scaling-out-policy \
  --policy-type StepScaling \
  --adjustment-type PercentChangeInCapacity \
  --min-adjustment-magnitude 1 \
  --metric-aggregation-type Average \
  --step-adjustments MetricIntervalLowerBound=0.0,MetricIntervalUpperBound=15.0,ScalingAdjustment=10 \
                     MetricIntervalLowerBound=15.0,MetricIntervalUpperBound=25.0,ScalingAdjustment=20 \
                     MetricIntervalLowerBound=25.0,ScalingAdjustment=30

 

Let's also create a scale in (decrease capacity) step scaling policy.

aws autoscaling put-scaling-policy \
  --auto-scaling-group-name my-auto-scaling-group  \
  --policy-name my-step-scale-in-policy \
  --policy-type StepScaling \
  --adjustment-type ChangeInCapacity \
  --step-adjustments MetricIntervalUpperBound=0.0,ScalingAdjustment=-2

 

The aws cloudwatch put-metric-alarm command to create a new cloudwatch alarm.

  • --period 300 and --evaulation-periods 1 means that the EC2 instance NetworkIn will be checked once every 300 seconds and there only needs to be a single occurrence (one evaulation period) where NetworkIn exceeds 1000000.0 (that's 1 million bytes or 1 MB of network in traffic) for the alarm to be triggered
  • The combination of --comparison-operator  GreaterThanThreshold and --threshold 1000000.0 and --statistic Average means that the alarm will be trigger when NetworkIn exceeds 1000000.0
  • Your Auto Scaling Policy will be invoked
  • A message will be published to your Simple Notification Service (SNS) Topic if the CPU Utilization of the EC2 instance exceeds 50% or if the Alarm has INSUFFICIENT_DATA.
aws cloudwatch put-metric-alarm \
--alarm-name high-network-in \
--alarm-description "Auto Scale when network in exceeds 1000000.0" \
--namespace AWS/EC2 \
--metric-name NetworkIn \
--comparison-operator GreaterThanThreshold \
--threshold 1000000.0 \
--statistic Average \
--period 300 \
--evaluation-periods 1 \
--treat-missing-data missing \
--datapoints-to-alarm 1 \
--dimensions "Name=InstanceId,Value=<your EC2 instance ID goes here, e.g. i-1234567890123456>" \
--alarm-actions <the ARN of your SNS Topic>,<the ARN of your stop out Auto Scaling Policy>,<the ARN of your stop in Auto Scaling Policy> \
--insufficient-data-actions <the ARN of your SNS topic>

 

For example.

aws cloudwatch put-metric-alarm \
--alarm-name high-network-in \
--alarm-description "Auto Scale when network in exceeds 1000000.0" \
--namespace AWS/EC2 \
--metric-name NetworkIn \
--comparison-operator GreaterThanThreshold \
--threshold 1000000.0 \
--statistic Average \
--period 300 \
--evaluation-periods 1 \
--treat-missing-data missing \
--datapoints-to-alarm 1 \
--dimensions "Name=InstanceId,Value=i-1234567890123456" \
--alarm-actions "arn:aws:sns:us-east-1:123456789012:my-topic",
                "arn:aws:autoscaling:us-east-1:123456789012:scalingPolicy:27e8fe33-8aa6-494c-9174-97ffdf42a5fe:autoScalingGroupName/my-auto-scaling-group:policyName/my-step-out-auto-scaling-policy",
                "arn:aws:autoscaling:us-east-1:123456789012:scalingPolicy:94fa28d6-1a8d-79ba-9182-f83de5f159ab:autoScalingGroupName/my-auto-scaling-group:policyName/my-step-in-auto-scaling-policy" \
--insufficient-data-actions "arn:aws:sns:us-east-1:123456789012:my-topic"

 




Did you find this article helpful?

If so, consider buying me a coffee over at Buy Me A Coffee



Comments


Add a Comment


Please enter b9b940 in the box below so that we can be sure you are a human.