Setting up custom metrics for autoscaling in AWS

In many cases, custom metrics may be useful to make more ad-hoc configuration of autoscaling in AWS. For example, a custom metric can allow an autoscaling group to react more rapidly on spikes or to take into account the health state of hosts for concurrency based metrics. ActiveConnectionCount per Healthy Host metric may be helpful to scale in an application, that may stop to respond to health-checks if it is fully loaded. In general, this metric may be used both to rapidly react on spikes, regardless of application type and may be more effective than CPU based and RequestPerHost metrics.

In this post, it is described how to manually set up a custom metric and autoscaling policy for an autoscaling group with an application load balancer and target group.

Prerequisites: autoscaling group with the target group and the application load balancer are working, you can request to load balancer and receive responses from target instances. Detailed logging is enabled for the application load balancer.

1) To create custom metric go to CloudWatch in AWS Console, click on Alarms -> Create Alarm

2) In Create New Alarm screen click on Select Metric

3) In Search field put ActiveConnectionCount and click on ApplicationELB -> Per AppELB Metrics, select one corresponding to your load balancer (if you have more than one load balancer - there will be shown one per load balancer)

4) Remove ActiveConnectionCount from Search field, to return to the root of Metric hierarchy, and remove everything from Search field

5) In Search field put HealthyHostCount , then click ApplicationELB -> Per AppELB, per TG Metrics

6) Select HealthyHostCount metric corresponding to the same load balancer that you selected on step 3

7) Go to Graphed metrics tab

8) Select Start with empty expression in the dropdown

9) In Details field of Expression1 put m1/m2

10) Select period 1 minute. Unselect m1 and m2, leave only e1 selected

11) Click Select metric

12) Put Metric name and description, i.e. ConnectionsPerHosts

13) Set number of datapoints

14) If you don't have notification list click New List and create one with your email

15) Select notification list

16) Click Create Alarm

17) You can see AutoScaling action, but it is better to set up the policy from autoscaling group screen.

18) On the Autoscaling groups screen go to Scaling Tab and click Add policy

19) Select metric you've created, it should be visible, but may appear with some delay, in my case sometimes it took hours to appear, so don't hurry to decide that something is wrong with the custom metric. Next click Create Policy button in the top right of the tab.

So, in such a manner, creating different Alarms, it is possible to configure very flexible policies.

Of course, int the most cases the policy should be configured programmatically and not from AWS console, but the described approach is useful to analyze and select the best metric.

Sources & More details:

AWS documentation