Case Study

Imagine you have a team that uses various cloud services from AWS, including AWS Lambda to run serverless functions. As the team progresses in their projects, they start noticing that their AWS costs are increasing. However, they are unsure which of their Lambda functions is causing most of these costs.

Challenge

The challenge here is to identify which Lambda function is consuming the most resources and, therefore, generating unnecessary costs. This can be complicated, especially if you have multiple functions and lack a proper monitoring system. Without effective monitoring, it’s easy to lose track of costs and performance metrics.

Monitoring Solution

To tackle this problem, we can implement a monitoring solution using Terraform, Amazon CloudWatch, and Amazon SNS. This will not only help you identify the functions generating excessive costs but also allow you to receive real-time notifications about any anomalies.

Project-Diagram:

Terraform Setup

First, make sure you have Terraform installed on your machine. You can download it from the official Terraform website. If you are a Mac user, this video will guide you on how to install Terraform on your computer:

Code Overview for Deploying the CloudWatch Dashboard using Terraform:

resource "aws_cloudwatch_dashboard" "lambda_cost_dashboard" {
  dashboard_name = "LambdaCostDashboard"
  dashboard_body = jsonencode({
    widgets = [
      {
        type = "metric",
        x = 0,
        y = 0,
        width = 24,
        height = 6,
        properties = {
          metrics = [
            ["AWS/Lambda", "Duration", "FunctionName", "${aws_lambda_function.my_lambda.function_name}"],  # Replace with your Lambda function name
            ["AWS/Lambda", "Cost", "FunctionName", "${aws_lambda_function.my_lambda.function_name}"]
          ],
          title = "Lambda Costs and Duration",
          period = 300,
          stat = "Average",
          view = "timeSeries",
          region = "us-east-1"
        }
      }
    ]
  })
}

resource "aws_cloudwatch_metric_alarm" "lambda_cost_alarm" {
  alarm_name          = "HighLambdaCost"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = "15"
  datapoints_to_alarm = "15"
  metric_name        = "Duration"
  namespace          = "AWS/Lambda"
  period             = "60"
  statistic          = "Average"
  threshold          = "25000"  # Change this threshold as needed
  treat_missing_data = "missing"
  dimensions = {
    FunctionName = "${aws_lambda_function.my_lambda.function_name}"  # Replace with your Lambda function name
  }

  alarm_description = "Alarm for high Lambda costs"
  alarm_actions     = [aws_sns_topic.lambda_cost_notifications.arn]
  insufficient_data_actions = []
}

This code will create a dashboard in CloudWatch that allows you to visualize the costs and duration of your Lambda function, along with an alarm that will notify you if the cost exceeds the defined threshold. You will also receive email notifications via SNS.

It is important to note that the email subscribed to Amazon SNS (Simple Notification Service) must be valid and confirmed to receive alerts. This means that if a message is sent to an SNS topic with a pending confirmation endpoint, it will not work as expected until the endpoint is confirmed.

You should look for a subscription confirmation email or check the topic in SNS to ensure that the subscription has been confirmed. Without this confirmation, the dashboard will not function properly, as alerts will not be sent.

Resource Optimization

It is crucial to optimize the memory configuration of your Lambda functions. It is recommended to start with at least 512 MB in most cases. This strategy not only improves performance but also helps minimize costs. Adjusting memory settings can significantly impact operational efficiency and the costs associated with using your Lambda functions.

Conclusion

Implementing a monitoring system for your AWS Lambda functions is essential for controlling costs and optimizing performance. By using Terraform, CloudWatch, and SNS, you can automate the creation of dashboards and alarms that will help you quickly identify any function generating excessive costs. With this solution, you will be better positioned to manage your cloud resources and ensure that your AWS Lambdas operate efficiently and cost-effectively.

Here you will find the GitHub repository containing the code and instructions.

GitHub Repository

Ready to streamline your AWS deployments?

Start using Terraform to manage your infrastructure as code. Experiment with the provided code and feel free to share your experiences in the comments. If you liked this article, follow me for more content on AWS and Terraform.

Your participation is valuable, and I would love to hear from you.