Let's say you have the following files on your Terraform server.

├── required_providers.tf
├── cloudwatch (directory)
│   ├── cloudwatch.tf
│   ├── provider.tf
│   ├── sns_topic.tf

required_providers.tf will almost always have this.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
    }
  }
}

Let's say provider.tf has the following. In this example, the "default" profile in /home/username/.aws/config and /home/username/.aws/credentials is being used. This assumes you have setup Terraform as described in Amazon Web Services (AWS) - Getting Started with Terraform.

provider "aws" {
  alias   = "default"
  profile = "default"
  region  = "default"
}

Before creating an alarm, you will almost always want to already have the Amazon Resource Number (ARN) of one of your AWS Simple Notification Service (SNS) Topics because typically, you will want to be notified when an alert fires. For example, perhaps you want to get an email when an alert fires. An AWS Simple Notification Service (SNS) Topics can be setup so that you get an email when the alert fires. Check out my article on Creating a Simple Notification Service (SNS) Topics using the AWS CLI.

~]$ aws sns create-topic --name my-topic 
{
    "TopicArn": "arn:aws:sns:us-east-1:123456789012:my-topic"
}

Then sns_topic.tf could have something like this, to get the ARN of the SNS topic usng the aws_sns_topic data module.

data "aws_sns_topic" "my_topic" {
  name = "my-topic"
}

And then users will subscribe to the Topic which can be done using the aws sns subscribe command.

 ~]$ aws sns subscribe --topic-arn arn:aws:sns:us-east-1:123456789012:my-topic --protocol email --notification-endpoint john.doe@example.com
{
    "SubscriptionArn": "pending confirmation"
}

And cloudwatch.tf could have something like this. In this example, the cloudwatch alarm will be triggered when an EC2 instance in my_target_group in application_load_balancer is unhealthy or has insufficient data.

• In this example, --period 120 and --evaulation-periods 1 means that the EC2 instance health will be checked once every 120 seconds and there only needs to be a single occurrence (one evaluation period) where the EC2 instance is unhealthy for the alarm to be triggered
• In this example, the combination of --comparison-operator  GreaterThanOrEqualToThreshold and --threshold 1 and --unit Sum means that the alarm will be trigger when EC2 instance is unhealthy

resource "aws_cloudwatch_metric_alarm" "UnHealthHostCount" {
  alarm_name                = "ALB-UnHealthHostCount"
  alarm_description         = "Application Load Balancer Target Group UnHealthHostCount"
  comparison_operator       = "GreaterThanOrEqualToThreshold"
  metric_name               = "UnHealthHostCount"
  namespace                 = "AWS/ApplicationELB"
  evaluation_periods        = 1
  period                    = 120
  statistic                 = "Sum"
  threshold                 = 1
  insufficient_data_actions = [data.aws_sns_topic.my_topic.arn]
  alarm_actions             = [data.aws_sns_topic.my_topic.arn]
  dimensions = {
    TargetGroup  = aws_lb_target_group.my_target_group.arn
    LoadBalancer = aws_lb.application_load_balancer.arn
  }
}

In this example, the cloudwatch alarm will be triggered when my-lambda-function goes into an Error state.

• In this example, --period 60 and --evaulation-periods 1 means that the Lambda Function will be checked once every 60 seconds and there only needs to be a single occurrence (one evaluation period) where the Lambda Function is in an Error state for the alarm to be triggered
• In this example, the combination of --comparison-operator  GreaterThanOrEqualToThreshold and --threshold 1 and --unit Sum means that the alarm will be trigger when there is a single occurrence in the evaulation period of the Lambda function being in an error state

data "aws_sns_topic" "my_topic" {
  name = "my-topic"
}

resource "aws_cloudwatch_metric_alarm" "lambda_errors" {
  alarm_name          = "my-lambda-function-errors"
  alarm_description   = "One or more errors were detected with my-lambda-function"
  comparison_operator = "GreaterThanOrEqualToThreshold"
  metric_name         = "Errors"
  namespace           = "AWS/Lambda"
  treat_missing_data  = "notBreaching"
  period              = 60 # 1 minute
  statistic           = "Sum"
  threshold           = 1
  evaluation_periods  = 1
  datapoints_to_alarm = 1

  dimensions = {
    FunctionName = "my-lamba-function"
  }

  alarm_actions = [data.aws_sns_topic.my_topic]
}

You may need to reissue the terraform init command.

~]# terraform init
Initializing the backend...
Initializing modules...
Initializing provider plugins...
Terraform has been successfully initialized!

The terraform plan command can be used to see what Terraform will try to do.

~]$ terraform plan
Terraform will perform the following actions:

  # aws_cloudwatch_log_group.my_log_group will be created
  + resource "aws_cloudwatch_log_group" "my_log_group" {
      + arn               = (known after apply)
      + id                = (known after apply)
      + name              = "my_log_group"
      + name_prefix       = (known after apply)
      + retention_in_days = 0
      + skip_destroy      = false
      + tags              = {
          + "Name" = "my_log_group"
        }
      + tags_all          = {
          + "Name" = "my_log_group"
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

The terraform apply command can be used to create the log group.

terraform apply -auto-approve