Amazon CloudWatch is AWS's monitoring and observability service. It collects operational data from AWS resources, applications and servers so that teams can measure performance, investigate failures, create alerts and view system health.
This guide provides AWS CloudWatch explained through a practical EC2 web server scenario. It covers metrics, logs, alarms, dashboards, AWS CLI commands and common troubleshooting steps.
If you are learning how monitoring fits into AWS architecture, the AWS Solutions Architect course covers CloudWatch alongside EC2, IAM, VPC, load balancing and other core services.
What is Amazon CloudWatch?
Amazon CloudWatch is a regional monitoring service that receives numeric measurements and log data from supported AWS services, applications and operating systems. It can visualise that data, evaluate alarm conditions and initiate actions when a system changes state.
A simple diagram-in-words looks like this:
AWS services, applications and servers
|
v
CloudWatch data collection
/ \
v v
Metrics CloudWatch Logs
| |
v v
Alarms Logs Insights queries
\ /
v v
Dashboards
|
v
Operators and automationCloudWatch answers operational questions such as:
- Is an EC2 instance using too much CPU?
- Is an application producing repeated HTTP 500 errors?
- Has an Application Load Balancer stopped receiving requests?
- Is a Lambda function failing or being throttled?
- Should an Auto Scaling group add instances?
CloudWatch should not be confused with AWS CloudTrail. CloudWatch monitors performance and operational data, while CloudTrail records AWS API activity for governance and auditing.
What are CloudWatch metrics?
CloudWatch metrics are time-ordered numeric data points that represent resource or application behaviour. Each metric belongs to a namespace and is identified by its metric name, dimensions and other attributes such as timestamp and unit.
For example, the AWS/EC2 namespace contains metrics published by Amazon EC2. The CPUUtilization metric can use InstanceId as a dimension so that CloudWatch can distinguish one instance from another.
| Metric component | Purpose | Example |
|---|---|---|
| Namespace | Groups related metrics | AWS/EC2 or CWAgent |
| Metric name | Identifies the measurement | CPUUtilization |
| Dimension | Identifies a resource or category | InstanceId=i-0123456789abcdef0 |
| Data point | Contains a value and timestamp | 72.4 at a specific time |
| Unit | Describes the value | Percent, Bytes or Count |
| Statistic | Aggregates data points | Average, Sum, Minimum or Maximum |
| Period | Defines the aggregation interval | 60 seconds or 300 seconds |
Many AWS services publish metrics automatically. EC2 basic monitoring commonly provides selected metrics at five-minute periods, while detailed monitoring provides one-minute periods for supported metrics.
Default EC2 metrics include CPU, network and instance status information. They do not include operating-system memory usage or normal filesystem utilisation because the EC2 service cannot see those values inside the guest operating system. The CloudWatch agent is required to collect them.
List available EC2 metrics for one instance with the AWS CLI:
aws cloudwatch list-metrics \
--namespace AWS/EC2 \
--metric-name CPUUtilization \
--dimensions Name=InstanceId,Value=i-0123456789abcdef0 \
--region ap-south-1A matching result includes the namespace, metric name and dimensions:
METRICS AWS/EC2 CPUUtilization
DIMENSIONS InstanceId i-0123456789abcdef0The presence of a metric definition does not guarantee that recent data exists. Use get-metric-data or the CloudWatch console to check timestamps and values.
What are custom metrics?
Custom metrics are measurements published by your application, script or CloudWatch agent. They are useful for values that AWS does not publish automatically, such as active sessions, queue depth inside an application or completed business transactions.
This command publishes a test metric:
aws cloudwatch put-metric-data \
--namespace NetworkRhinos/Lab \
--metric-name ActiveSessions \
--dimensions Environment=Training \
--value 14 \
--unit Count \
--region ap-south-1Use dimensions carefully. Every unique dimension combination creates a separate metric, so uncontrolled values such as request IDs can create unnecessary cost and complexity.
How do CloudWatch Logs work?
CloudWatch Logs stores text-based events produced by applications, operating systems and AWS services. Events are organised into log groups and log streams, and retention can be configured separately for each log group.
The main hierarchy is:
Log group: /training/web/access
|
+-- Log stream: i-0123456789abcdef0
| +-- 2026-09-29T10:10:01Z GET /index.html 200
| +-- 2026-09-29T10:10:05Z GET /health 200
|
+-- Log stream: i-0fedcba9876543210
+-- 2026-09-29T10:10:07Z GET /login 500A log event contains a timestamp and message. A log stream normally represents one source, such as an EC2 instance or container. A log group gathers streams that share retention, access and monitoring settings.
CloudWatch Logs receives data from services such as Lambda, API Gateway and VPC Flow Logs when logging is enabled. EC2 application and operating-system files require an agent or application integration.
Use AWS CLI v2 to follow recent log events:
aws logs tail /training/web/access \
--since 15m \
--follow \
--region ap-south-1Set a retention period instead of leaving training or temporary logs indefinitely:
aws logs put-retention-policy \
--log-group-name /training/web/access \
--retention-in-days 30 \
--region ap-south-1Choose retention according to operational, security and compliance requirements. Longer retention increases stored data and can affect cost.
How does CloudWatch Logs Insights search logs?
CloudWatch Logs Insights is a query tool for searching and analysing log events across selected log groups. It can filter errors, calculate counts, parse fields and sort results without manually downloading files.
The following query finds common server errors:
fields @timestamp, @message
| filter @message like /ERROR|Exception|HTTP 5[0-9][0-9]/
| sort @timestamp desc
| limit 50Another query counts messages in five-minute intervals:
fields @timestamp
| stats count(*) as event_count by bin(5m)
| sort @timestamp descSelect the smallest practical time range before running a query. Logs Insights processes the selected log data, so broad queries can take longer and scan more data.
How do CloudWatch alarms work?
A CloudWatch alarm evaluates a metric or metric-math expression against a defined condition. Its state can be OK, ALARM or INSUFFICIENT_DATA, and actions can run when the state changes.
An alarm definition includes the metric, statistic, period, threshold, comparison operator and evaluation settings. For example, an alarm can enter ALARM when average memory utilisation is above 80 percent for at least three of the last five one-minute periods.
CWAgent mem_used_percent
|
v
Average each 60-second period
|
v
Are 3 of the last 5 values above 80?
/ \
No Yes
| |
v v
OK ALARM
|
v
SNS notificationCreate that alarm with the AWS CLI:
aws cloudwatch put-metric-alarm \
--alarm-name training-web-high-memory \
--namespace CWAgent \
--metric-name mem_used_percent \
--dimensions Name=InstanceId,Value=i-0123456789abcdef0 \
--statistic Average \
--period 60 \
--evaluation-periods 5 \
--datapoints-to-alarm 3 \
--threshold 80 \
--comparison-operator GreaterThanThreshold \
--treat-missing-data missing \
--alarm-actions arn:aws:sns:ap-south-1:123456789012:operations-alerts \
--region ap-south-1The SNS topic must already exist, and subscriptions must be confirmed where confirmation is required. The caller also needs permission to create the alarm and use the configured action.
Alarm actions can notify Amazon SNS, interact with supported EC2 actions or participate in Auto Scaling policies. Composite alarms combine other alarm states and can reduce repeated notifications during one wider incident.
Missing-data behaviour deserves attention. Treating missing data as breaching may detect a failed data source, but it may also produce false alarms for metrics that appear only when an event occurs.
What are CloudWatch dashboards?
CloudWatch dashboards are configurable pages containing metric graphs, numbers, text, alarm status and supported query widgets. They provide a shared operational view but do not collect data or create alarms by themselves.
A practical web application dashboard might contain:
| Dashboard row | Suggested widgets |
|---|---|
| User traffic | Load balancer request count and target response time |
| Compute health | EC2 CPU, memory and status-check failures |
| Application health | HTTP 4xx, HTTP 5xx and error log count |
| Capacity | Auto Scaling desired, minimum and maximum capacity |
| Alerts | Current state of critical alarms |
Dashboards can present metrics from multiple AWS Regions. Cross-account information can also be centralised when CloudWatch cross-account observability is configured with appropriate permissions.
Choose graphs that support an operational decision. A dashboard with many unrelated metrics is harder to use than one organised around traffic, errors, latency and resource saturation.
How can you monitor an EC2 web server with CloudWatch?
A useful CloudWatch lab is to monitor an EC2-hosted web server's CPU, memory and access logs, then create a memory alarm. The instance needs network access to CloudWatch endpoints and an IAM role with suitable agent permissions.
The managed CloudWatchAgentServerPolicy is commonly attached to an EC2 instance role for this lab. In production, review the actions and resources and apply least-privilege permissions where practical.
Step 1: Install the CloudWatch agent on Ubuntu
wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb
sudo dpkg -i amazon-cloudwatch-agent.debConfirm that the package is installed:
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a statusBefore configuration, the status may show that the agent is stopped.
Step 2: Create the agent configuration
sudo tee /opt/aws/amazon-cloudwatch-agent/etc/web-lab.json > /dev/null <<'EOF'
{
"agent": {
"metrics_collection_interval": 60
},
"metrics": {
"namespace": "CWAgent",
"append_dimensions": {
"InstanceId": "${aws:InstanceId}"
},
"metrics_collected": {
"mem": {
"measurement": ["mem_used_percent"]
},
"disk": {
"measurement": ["used_percent"],
"resources": ["/"]
}
}
},
"logs": {
"logs_collected": {
"files": {
"collect_list": [
{
"file_path": "/var/log/nginx/access.log",
"log_group_name": "/training/web/access",
"log_stream_name": "{instance_id}",
"retention_in_days": 30
}
]
}
}
}
}
EOFThis configuration sends memory and root-filesystem metrics to the CWAgent namespace. It also sends the Nginx access log to /training/web/access.
Step 3: Start the agent
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
-a fetch-config \
-m ec2 \
-s \
-c file:/opt/aws/amazon-cloudwatch-agent/etc/web-lab.jsonCheck its status:
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a statusA healthy response includes values similar to:
status: running
starttime: 2026-09-29T10:15:20+00:00
configstatus: configuredAllow several minutes for the first data points to appear. Then check the CWAgent namespace and the log group in the same Region as the instance.
For broader automation practice, CloudWatch alarms and dashboards are also relevant to deployment workflows covered in an AWS DevOps course. For serverless monitoring, see AWS Lambda triggers, runtimes and use cases, including the role of function logs and metrics.
How do you troubleshoot missing CloudWatch data?
Missing data is usually caused by the wrong Region, incorrect dimensions, agent failure, IAM denial, network connectivity or a mismatch between the configured source and the actual file or metric. Troubleshoot from the data producer towards CloudWatch instead of repeatedly recreating the dashboard.
1. Confirm the AWS Region
Metrics and log groups are regional. Verify the CLI configuration and pass --region explicitly during testing.
aws configure get region
aws sts get-caller-identity2. Check the agent service and diagnostic log
sudo systemctl status amazon-cloudwatch-agent
sudo tail -n 100 /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.logLook for credential errors, access denials, invalid configuration fields or connection failures.
3. Verify the instance role
From EC2, confirm that an IAM instance profile is attached. Avoid placing long-term IAM access keys in agent configuration files.
An AccessDenied message means credentials were found but do not permit the requested CloudWatch or Logs action. A missing-credentials message usually means the instance role is absent or unavailable.
4. Verify the source log file
sudo ls -l /var/log/nginx/access.log
sudo tail -n 10 /var/log/nginx/access.logIf the file does not exist, confirm that Nginx is installed, running and configured to write to that path. Also generate a request so that the file contains new events.
5. Match alarm dimensions exactly
A metric with InstanceId and path dimensions is different from a metric containing only InstanceId. Use list-metrics to inspect the published dimension set before creating an alarm.
6. Review missing-data settings
An alarm in INSUFFICIENT_DATA may not have enough periods to evaluate, or the source may have stopped publishing. Check the metric graph, period, evaluation window and treat-missing-data setting before changing the threshold.
What CloudWatch practices should teams follow?
Effective monitoring begins with a small set of actionable signals rather than every available metric. Use consistent names, configure retention, protect log data and connect each important alarm to a documented response.
Recommended practices include:
- Create alarms for symptoms that require action, not harmless short spikes.
- Use multiple evaluation periods to reduce transient alerts.
- Separate warning and critical thresholds where this helps operations.
- Encrypt sensitive log groups when required and control access with IAM.
- Avoid writing passwords, access keys or personal data into logs.
- Set log retention according to operational and compliance needs.
- Use tags and naming standards for dashboards and alarms.
- Review custom metric dimensions to control cardinality.
- Test alarm actions safely before relying on them during an incident.
- Monitor estimated usage and cost as logging volume grows.
Frequently asked questions
Is Amazon CloudWatch enabled automatically?
Many AWS services automatically publish selected metrics to CloudWatch, but the available metrics and periods differ by service. Application logs, operating-system memory and filesystem metrics normally require explicit configuration or the CloudWatch agent.
What is the difference between CloudWatch metrics and logs?
Metrics are numeric time-series values designed for graphs, statistics and alarms. Logs are timestamped text events used for detailed investigation, searching and field analysis.
Can CloudWatch monitor servers outside AWS?
Yes. The CloudWatch agent can run on supported on-premises or other non-EC2 servers, provided it has AWS credentials, network connectivity and a valid agent configuration.
Why is a CloudWatch alarm showing insufficient data?
The alarm has not received enough matching data points for evaluation, or the metric has stopped publishing. Check the Region, dimensions, period, source health and missing-data configuration.
Does CloudWatch replace CloudTrail?
No. CloudWatch focuses on monitoring, logs, metrics, alarms and operational visibility, while CloudTrail records AWS API and account activity for auditing and investigation.
Can one CloudWatch dashboard show multiple Regions?
Yes. A dashboard can contain widgets that reference metrics from different AWS Regions. Cross-account views require CloudWatch cross-account observability and suitable permissions.
Summary
CloudWatch turns AWS and application data into operational visibility. Metrics measure behaviour, logs preserve detailed events, alarms evaluate conditions, and dashboards bring the important signals into one view.
A practical learning path is to install the CloudWatch agent on EC2, publish memory and disk metrics, collect a real application log, create an alarm and troubleshoot the complete flow. This demonstrates how monitoring supports availability, performance and incident response.
To practise CloudWatch with EC2, IAM, VPC, load balancing and architecture labs, enquire about upcoming batch details for the AWS Solutions Architect course.
*Reviewed by Network Rhinos AWS and cloud trainers.*