> For the complete documentation index, see [llms.txt](https://ysfang82.gitbook.io/development-notes/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ysfang82.gitbook.io/development-notes/aws-notes/service/management-tools/cloudwatch.md).

# CloudWatch

**Introduction**

* Monitoring operational and performance metrics for your AWS cloud resources and applications.

**Feature**

* Dashboard
  * Display metrics and alarms for AWS resources (Not for configuration of AWS Organizations)
  * Can show metrics of **multiple regions**
* Metrics:
  * Provided by many AWS services
    * Can create custom metrics: standard resolution: 1 minute, high resolution: 1 second
  * EC2
    * Standard: 5 minutes, Detailed monitoring: 1 minute
    * **RAM is not a built-in metric (Use CloudWatch Unified Agent)**
  * **RI utilization is not tracked by CloudWatch.**
    * Create a new reservation budget in AWS Budgets service. [Set the reservation budget](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/budgets-create.html#create-reservation-budget) type to be RI Utilization and configure the utilization threshold.
* Alarms:
  * Can trigger actions: EC2 action (reboot, stop, terminate, recover), Auto scaling, SNS
  * Alarm events can be intercepted by CloudWatch Events
* Events
  * Can intercept from AWS services (ex. EC2 Start, CodeBuild failure, CloudWatch Alarms, Trusted Advisor, CloudTrail API calls, etc.)
  * Can trigger: 
    * Compute: Lambda, Batch, ECS task
    * Orchestration: Step Functions, CodePipeline, CodeBuild
    * Integration: SQS, SNS, Kinesis Data streams, Kinesis Firehose
    * Maintenance: SSM, EC2 actions
  * [Rule Expression](https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html)
    * (minute, hour, day of month, month, day of week, year)
* Logs
  * Sources:
    * SDK, CloudWatch Logs Agent, CloudWatch Unified Agent
    * Elastic Beanstalk: collection of logs from application
    * ECS: collection from containers
    * AWS Lambda: collection from function logs
    * VPC Flow Logs: VPC specific logs
    * API Gateway
    * CloudTrail based on filter
    * CloudWatch Log agents (ex. on EC2 machines)
    * Route53: log DNS queries
  * Destination:
    * S3 (export)
      * The bucket must be SSE-S3, not SSE-KMS.
      * Log data can take up to 12 hours to become available for export
        * Use CloudWatch Logs Insights or Logs Subscriptions for real time processing
    * Kinesis Data Stream / Firehose
    * Lambda
    * ElasticSearch
  * Log groups: arbitrary name, usually representing an application
  * Log stream: instances within application / log files /containers
  * Can define log expiration policies (never expire / 30 days, etc.)
  * Optional KMS encryption
  * CloudWatch Logs Insights can be used to query logs and add queries to CloudWatch Dashboards.
  * Logs Subscription
    * Works with a filter by Lambda
    * Send logs out with:
      * Lambda (real time)
      * Kinesis Data Streams (real time)
      * Kinesis Firehose (near real time)
  * Logs aggregation with multi-account & multi-region
    * By each account, set up Subscription Filter to send logs to central Kinesis Data Streams, then send logs to Kinesis Firehose to write logs to S3. 
  * Logs Agent & Unified Agent
    * For virtual servers (EC2 instances, on-premise servers,...)
    * CloudWatch Logs Agent
      * Old version of the agent
      * Can only send to CloudWatch Logs
    * CloudWatch Unified Agent
      * Collect additional system-level metrics such as RAM, processes, etc.
      * Collect logs to send to CloudWatch Logs
      * Centralized configuration using SSM Parameter Store
    * Batch sends capabilities and settings:
      * batch\_count: number of log events to send (default: 10,000, minimum: 1)
      * batch\_duration: duration of batching for log events (default / minimum: 5000ms)
      * batch\_size: max size of log events in a batch (default / max: 1 MB)
    * Both agents cannot send logs to Kinesis