Amazon CloudWatch Overview

• Definition: A monitoring service for AWS cloud resources and applications running on AWS.
• Core Functions: Collects and tracks metrics, collects and monitors log files, and sets alarms.
• Benefits:
  • Gain system-wide visibility into resource utilization.
  • Monitor application performance.
  • Monitor operational health.
• Access: Via API, CLI, AWS SDKs, and AWS Management Console.
• Integrates with AWS IAM for access control.
• Can automatically react to changes in your AWS resources.
• Monitors resources such as: EC2 instances, DynamoDB tables, RDS DB instances, custom metrics, and log files.

Metrics

Metrics Basics

• Fundamental concept in CloudWatch.
• Represents a time-ordered set of data points published to CloudWatch.
• AWS services automatically send metrics to CloudWatch. You can also send your own custom metrics.
• Metrics exist within a region.
• Metrics cannot be deleted but automatically expire after 15 months.
• Uniquely defined by a name, a namespace, and zero or more dimensions.

Metric Data Retention

• Data points < 60 seconds (high-resolution): Available for 3 hours.
• Data points 60 seconds (1 minute): Available for 15 days.
• Data points 300 seconds (5 minutes): Available for 63 days.
• Data points 3600 seconds (1 hour): Available for 455 days (15 months).

Custom Metrics

• Publish your own metrics to CloudWatch using AWS CLI or API (PutMetricData).
• View statistical graphs in the console.
• Each data point has an associated timestamp.
• Can publish an aggregated set of data points called a statistic set.
• Every PutMetricData call for a custom metric is charged.

High-Resolution Metrics

• Standard Resolution: 1-minute granularity (default for AWS services).
• High Resolution: 1-second granularity (for custom metrics).
• When published, stored with 1-second resolution. Can be retrieved with periods of 1, 5, 10, 30 seconds, or any multiple of 60 seconds.
• Provides more immediate insight into sub-minute activity.
• Higher charges for more frequent PutMetricData calls and for high-resolution alarms (10 or 30-second periods).

Namespaces

• A container for CloudWatch metrics.
• Isolates metrics from different applications to prevent mistaken aggregation.
• Examples: AWS/EC2, AWS/S3, AWS/Lambda, MyApplication/WebTier.

Dimensions

• Further clarifies what a metric is and what data it stores.
• Up to 10 dimensions per metric, each defined by a name and value pair.
• Syntax for CLI:
  • put-metric-data: MyName=MyValue
  • get-metric-statistics or put-metric-alarm: Name=MyName,Value=MyValue
• If a metric has multiple dimensions, you must specify a value for every defined dimension when using get-metric-statistics.

Statistics

• Aggregations of metric data points over specified periods.
• Provided by CloudWatch based on custom data or AWS service data.
• Types:
  • Minimum: Lowest value.
  • Maximum: Highest value.
  • Sum: Total of all values.
  • Average: Sum / SampleCount.
  • SampleCount: Number of data points.
  • pNN.NN (Percentiles): Value of the specified percentile (e.g., p99 for 99th percentile). Not available for negative values.

CloudWatch Alarms

• Monitor a single metric over a specified time period.
• Initiate one or more specified actions based on the metric's value relative to a threshold.
• Actions include: Notification to an Amazon SNS topic or an Auto Scaling policy.
• Can be added to dashboards.
• Alarms invoke actions for sustained state changes only (not just being in a state). The state must be maintained for a specified number of periods.
• High-resolution alarms can have periods of 10 or 30 seconds (higher charge).

CloudWatch Logs

• Purpose: Monitor and troubleshoot systems and applications using existing system, application, and custom log files.
• Centralizes logs from EC2 instances, AWS CloudTrail, Route 53, and other sources.
• Features:
  • Monitor logs from EC2 instances (application and system logs, can trigger notifications).
  • Monitor CloudTrail Logged Events (create alarms based on API activity).
  • Log Retention: By default, logs are retained indefinitely. Configurable per log group from 1 day to 10 years.
  • Used for real-time application/system monitoring and long-term log retention.
  • Metric Filters: Scan log data to generate CloudWatch metrics. (Exam Tip!)
  • Subscription Filters: Send log data to a subscriber (e.g., Lambda for real-time processing, Kinesis Data Streams).
  • Export logs to S3 (via CreateExportTask, can take hours).
  • For near real-time or persistent logs, consider Kinesis Firehose for supported destinations, or Kinesis Data Streams with Lambda/KCL for custom real-time processing.

CloudWatch Logs Agent (Unified Agent)

• Provides an automated way to send log data and system-level metrics to CloudWatch Logs from EC2 instances and on-premises servers.
• Unified Agent Capabilities:
  • Collects in-guest metrics (e.g., memory, disk utilization) from EC2 instances across OS.
  • Collects system-level metrics from on-premises servers (hybrid environments).
  • Retrieves custom metrics from applications/services using StatsD (Linux/Windows) and collectd (Linux only) protocols.

CloudWatch Events (Amazon EventBridge)

• Definition: Delivers a near real-time stream of system events that describe changes in AWS resources.
• Can schedule automated actions that self-trigger at certain times using cron or rate expressions.
• Can match events (based on patterns) and route them to one or more target functions or streams.
• Targets include: EC2 instances, Lambda functions, Kinesis Data Streams, Kinesis Data Firehose, CloudWatch Logs, ECS tasks, Systems Manager Run Command/Automation, AWS Batch jobs, Step Functions state machines, CodePipeline, CodeBuild, SNS topics, SQS queues.
• Note: CloudWatch Events has been largely superseded by Amazon EventBridge, which offers enhanced features like event buses, schema registry, and SaaS integrations. For exam purposes, often "CloudWatch Events" refers to the core functionality now part of EventBridge.

Useful API Actions (for Developer Associate Exam)

• GetMetricData: Retrieve up to 500 different metrics in a single request.
• PutMetricData: Publishes metric data points to CloudWatch. Creates the metric if it doesn't exist.
• GetMetricStatistics: Gets aggregated statistics (Min, Max, Sum, Avg, SampleCount, Percentiles) for a specified metric over a period. Max 1,440 data points per call.
• PutMetricAlarm: Creates or updates an alarm associated with a metric, metric math expression, or anomaly detection model.

CloudWatch vs. CloudTrail Comparison

Important Exam Tips & Tricks

• "How is my application performing?" → CloudWatch.
• "Who did what, when, where?" → CloudTrail.
• Metrics Granularity: Standard (1 min) vs. High-Resolution (1 sec). Know the cost implications of high-resolution metrics and alarms.
• Metrics Retention: Remember the 15-month retention for metrics, with decreasing granularity over time.
• Namespaces & Dimensions: Crucial for organizing and filtering metrics. Understand how to use them to uniquely identify a metric.
• Alarms: Alarms only trigger for sustained state changes. They don't just trigger if a metric is above a threshold, but if it stays above for a defined period.
• CloudWatch Logs:
  • Centralized logging.
  • Metric Filters: Extract metrics from logs.
  • Subscription Filters: Stream logs to other services (Lambda, Kinesis).
  • Log retention is configurable per log group.
• Unified CloudWatch Agent: Collects both logs and in-guest metrics (memory, disk, etc.) from EC2 and on-premises servers. This is key for getting OS-level visibility beyond basic EC2 metrics.
• CloudWatch Events (EventBridge): Focus on event-driven architectures and scheduled tasks (cron/rate expressions). It's a router for events.
• API Calls: Be familiar with PutMetricData (send data) and GetMetricStatistics (retrieve aggregated data).
• Custom Metrics Cost: Be aware that publishing custom metrics incurs costs, especially high-resolution ones.