Let's say you have the following files on your Terraform server.

├── required_providers.tf
├── cloudwatch (directory)
│   ├── cloudwatch.tf
│   ├── provider.tf
│   ├── sns_topic.tf

required_providers.tf will almost always have this.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
    }
  }
}

Let's say provider.tf has the following. In this example, the "default" profile in /home/username/.aws/config and /home/username/.aws/credentials is being used. This assumes you have setup Terraform as described in Amazon Web Services (AWS) - Getting Started with Terraform.

provider "aws" {
  alias   = "default"
  profile = "default"
  region  = "default"
}

Before creating an alarm, you will need the Amazon Resource Number (ARN) of one of your AWS Simple Notification Service (SNS) Topics because typically, you will want to be contacted somehow when an alert fires. For example, perhaps you want to get an email when an alert fires. An AWS Simple Notification Service (SNS) Topics can be setup so that you get an email when the alert fires. Check out my article on Creating a Simple Notification Service (SNS) Topics using the AWS CLI.

~]$ aws sns create-topic --name my-topic 
{
    "TopicArn": "arn:aws:sns:us-east-1:123456789012:my-topic"
}

Then sns_topic.tf could have something like this, to get the ARN of the SNS topic usng the aws_sns_topic data module.

data "aws_sns_topic" "my_topic" {
  name = "my-topic"
}

And then users will subscribe to the Topic which can be done using the aws sns subscribe command.

 ~]$ aws sns subscribe --topic-arn arn:aws:sns:us-east-1:123456789012:my-topic --protocol email --notification-endpoint john.doe@example.com
{
    "SubscriptionArn": "pending confirmation"
}

And cloudwatch.tf could have something like this. In this example, the cloudwatch alarm will be triggered when an EC2 instance in my_target_group in application_load_balancer is unhealthy or has insufficient data.

• In this example, --period 120 and --evaulation-periods 1 means that the EC2 instance health will be checked once every 120 seconds and there only needs to be a single occurrence (one evaulation period) where the EC2 instance is unhealthy for the alarm to be triggered
• In this example, the combination of --comparison-operator  GreaterThanOrEqualToThreshold and --threshold 1 and --unit Sum means that the alarm will be trigger when EC2 instance is unhealthy

resource "aws_cloudwatch_metric_alarm" "UnHealthHostCount" {
  alarm_name                = "ALB-UnHealthHostCount"
  alarm_description         = "Application Load Balancer Target Group UnHealthHostCount"
  comparison_operator       = "GreaterThanOrEqualToThreshold"
  metric_name               = "UnHealthHostCount"
  namespace                 = "AWS/ApplicationELB"
  evaluation_periods        = 1
  period                    = 120
  statistic                 = "Sum"
  threshold                 = 1
  insufficient_data_actions = [data.aws_sns_topic.my_topic.arn]
  alarm_actions             = [data.aws_sns_topic.my_topic.arn]
  dimensions = {
    TargetGroup  = aws_lb_target_group.my_target_group.arn
    LoadBalancer = aws_lb.application_load_balancer.arn
  }
}

You may need to reissue the terraform init command.

~]# terraform init
Initializing the backend...
Initializing modules...
Initializing provider plugins...
Terraform has been successfully initialized!

The terraform plan command can be used to see what Terraform will try to do.

~]$ terraform plan
Terraform will perform the following actions:

  # aws_cloudwatch_log_group.my_log_group will be created
  + resource "aws_cloudwatch_log_group" "my_log_group" {
      + arn               = (known after apply)
      + id                = (known after apply)
      + name              = "my_log_group"
      + name_prefix       = (known after apply)
      + retention_in_days = 0
      + skip_destroy      = false
      + tags              = {
          + "Name" = "my_log_group"
        }
      + tags_all          = {
          + "Name" = "my_log_group"
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

The terraform apply command can be used to create the log group.

terraform apply -auto-approve