Choosing Metrics

AWS takes the approach of “working backward” from the customer. You can accomplish this by starting with the customer and tracing the underlying components that affect the customer’s experience. This provides a foundation for identifying which metrics to monitor, as they correlate directly to the customer experience. Frequently, the top characteristics that directly affect the customer experience are performance and cost. Changes to either have a direct impact on how customers perceive the software they use.

Deciding which metrics to monitor requires that you answer several crucial questions.

Built-In Metrics

A metric is a set of time-series data points that you publish to CloudWatch. For example, a commonly monitored metric for Amazon EC2 instances is CPU utilization. Data points can come from multiple systems, both AWS and on-premises. You can also define custom metrics based on data specific to your system. A metric is identified uniquely by a namespace, a name, and zero or more dimensions.

Data Points

When data is published to CloudWatch, it is pushed in sets of data points. Each data point contains information such as the timestamp, value, and unit of measurement.

Statistics

CloudWatch provides statistics based on metric data provided to the service. Statistics are aggregations of data points over specified periods of time for specified metrics. A period is the length of time, in seconds. Periods can be defined in values of 1, 5, 10, 30, or any multiple of 60 seconds (up to 86,400 seconds, or 1 day). The available statistics in CloudWatch include the following:

• Minimum (Min), the lowest value recorded over the specified period
• Maximum (Max), the highest value recorded over the specified period
• Sum, the total value of the samples added together over the specified period
• Average (Avg), the Sum divided by the SampleCount over the specified period
• SampleCount, the number of data points used in the calculation over the specified period
• pNN, percentile statistics for tracking metric outliers

Statistics can be used to gain insight into the health of your application and to help you determine the correct settings for various configurations. For example, you may want to implement automatic scaling on your fleet of Amazon EC2 instances in order to avoid having to launch and terminate instances manually. To do so, you must configure an Auto Scaling group. Configuration settings for an Auto Scaling group include the minimum, desired, and maximum number of instances to run in your account. By monitoring statistics over time, you can determine the minimum and maximum number of instances needed to support the average, minimum, and maximum workload.

CloudWatch statistics provide a powerful way to process large amounts of metrics at scale and present insightful data that is easy to consume. Now that you understand how CloudWatch metrics work and are organized, explore the metrics available.

Aggregations

CloudWatch aggregates metrics according to the period of time you specify when retrieving statistics. When you request this statistic, you also can have CloudWatch filter the data points based on the dimensions of the metrics. For example, in Amazon DynamoDB, metrics are fetched across all DynamoDB operations. You can specify a filter on the dimension operations to exclude specific operations, such as GetItem requests. CloudWatch does not aggregate data across regions.

Available Metrics

Table 15.1 describes the available metrics for Elastic Load Balancing resources. To discover all of the available metrics, refer to the AWS documentation.

Table 15.2 describes the available Amazon EC2 metrics.

Table 15.3 describes the AWS Auto Scaling group metrics.

Table 15.4 describes the Amazon Simple Storage Service (Amazon S3) metrics.

Table 15.5 describes the Amazon DynamoDB metrics.

Table 15.6 describes the Amazon API Gateway metrics.

Table 15.7 describes the AWS Lambda metrics.

Table 15.8 describes the Amazon Simple Queue Service (Amazon SQS) metrics.

Table 15.9 describes the Amazon Simple Notification Service (Amazon SNS) metrics.

Custom Metrics

In addition to the built-in metrics that AWS provides, CloudWatch also supports custom metrics that you can publish from your systems. This section includes some commands that you can use to publish metrics to CloudWatch.

aws cloudwatch put-metric-data 
      --metric-name PageViewCount 
      --namespace MyService 
      --value 2 
      --timestamp 2018-10-20T12:00:00.000Z
 
aws cloudwatch put-metric-data 
      --metric-name PageViewCount 
      --namespace MyService 
      --value 4 
      --timestamp 2018-10-20T12:00:01.000Z
 
aws cloudwatch put-metric-data 
      --metric-name PageViewCount 
      --namespace MyService 
      --value 5 
      --timestamp 2018-10-20T12:00:02.000Z

aws cloudwatch put-metric-data 
      --metric-name PageViewCount 
      --namespace MyService 
      --statistic-values Sum=11,Minimum=2,Maximum=5,SampleCount=3 
      --timestamp 2018-10-14T12:00:00.000Z

aws cloudwatch put-metric-data 
      --metric-name PageViewCount 
      --namespace MyService 
      --value 0 
      --timestamp 2018-10-14T12:00:00.000Z

aws cloudwatch get-metric-statistics 
      --namespace MyService 
      --metric-name PageViewCount 
      --statistics "Sum" "Maximum" "Minimum" "Average" "SampleCount" 
      --start-time 2018-10-20T12:00:00.000Z 
      --end-time 2018-10-20T12:05:00.000Z 
      --period 60

{
    "Datapoints": [
        {
            "SampleCount": 3.0,
            "Timestamp": "2016-10-20T12:00:00Z",
            "Average": 3.6666666666666665,
            "Maximum": 5.0,
            "Minimum": 2.0,
            "Sum": 11.0,
            "Unit": "None"
        }
    ],
    "Label": "PageViewCount"
}

Log Aggregation

Setting up centralized logging with CloudWatch Logs is a straightforward process. The first step is to install and configure the CloudWatch agent, which is used to collect custom logs and metrics from Amazon EC2 instances or on-premises servers. You can choose which log files you want to ingest by pointing to the locations using a JavaScript Object Notation (JSON) configuration file. The second step is to configure AWS Identity and Access Management (IAM) roles or users to grant permission for the agent to publish logs into CloudWatch. In addition to the CloudWatch agent, you can also send metrics to CloudWatch using the AWS CLI, AWS SDK, or AWS API.

Because you are collecting logs from multiple sources, CloudWatch organizes your logs into three conceptual levels: groups, streams, and events.

Log Searches

With centralized logging on CloudWatch Logs, you do not need to search through hundreds of individual servers to find a problem. After logs are ingested into CloudWatch Logs, you can search for logs through a central location using metric filters.

{
    "user": {
        "id": 1,
        "email": "[email protected]"
              
    },
    "users": [
        {
         "id": 2,
         "email": "[email protected]"
              
        },
        {
         "id": 3,
         "email": "[email protected]"
              
        }
    ],
    "actions": [
        "GET",
        "PUT",
        "DELETE"
    ],
    "coordinates": [
        [0, 1, 2],
        [4, 5, 6],
        [7, 8, 9]
    ]
}
 

Log Processing

Instead of having to write additional code to add monitoring to your application, CloudWatch can process logs that you already generate and provide valuable metrics. Using the example from the previous section, the same metric filter can be used to generate metrics corresponding to the number of occurrences of the term ERROR in your logs.

Using Amazon CloudWatch Alarms

When you create an alarm, specify three settings that determine when the alarm should change states: the threshold, period, and data points on which you want to notify, as described in Table 15.14.

Figure 15.3 illustrates how an alarm works based on configuration settings.

The figure illustrates a threshold configured to the value 3 (in blue), a period set to 3, and data points in red. Notice how the settings drive the alarm occurrence. Even though the data points breach the threshold after the third period, it is not sustained for the required three periods to be in an ALARM state. Only after the fifth period would the alarm change to an ALARM state (the upper threshold is breached for three periods). Between the fifth and sixth period, the data points drop below the threshold. However, because the state has not dropped below the threshold for three periods, it does not change to an OK state until the eighth period. It remains in the OK state past the ninth period because three consecutive periods exceeding the threshold are necessary for the alarm state to change.

Alarms can trigger Amazon EC2 actions and EC2 Auto Scaling actions. CloudWatch can leverage Amazon SNS or Amazon SQS for alarm state notifications, both of which provide numerous integrations with other AWS services.

Management Events

Management events give insight into operations performed on AWS resources, such as the following examples:

• Configuring security: An example is attaching a policy to an IAM role.
• Configuring routing rules: An example is adding inbound security group rules.

Data Events

Data events give insight into operations that store data in (or extract data from) AWS resources, such as the following examples:

• Amazon S3 object activity: Examples are GetObject and PutObject operations.
• AWS Lambda function executions: These use the InvokeFunction operation.

By default, CloudTrail tracks the last 90 days of API history for management events. The following is example output for a CloudTrail event:

{
   "Records": [{
      "eventVersion": "1.01",
      "userIdentity": {
         "type": "IAMUser",
         "principalId": "AIDAJDPLRKLG7UEXAMPLE",
         "arn": "arn:aws:iam::123456789012:user/Alice",
         "accountId": "123456789012",
         "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
         "userName": "Alice",
         "sessionContext": {
            "attributes": {
               "mfaAuthenticated": "false",
               "creationDate": "2014-03-18T14:29:23Z"
            }
         }
      },
      "eventTime": "2014-03-18T14:30:07Z",
      "eventSource": "cloudtrail.amazonaws.com",
      "eventName": "StartLogging",
      "awsRegion": "us-west-2",
      "sourceIPAddress": "198.162.198.64",
      "userAgent": "signin.amazonaws.com",
      "requestParameters": {
         "name": "Default"
      },
      "responseElements": null,
      "requestID": "cdc73f9d-aea9-11e3-9d5a-835b769c0d9c",
      "eventID": "3074414d-c626-42aa-984b-68ff152d6ab7"
   },
    ... additional entries ...
   ]
 

This event provides the following information:

• The user who made the request from the userIdentityField. In this example, it is the IAM user Alice.
• When the request was made (the eventTime). In this case, it is 2014-03-18T14:30:07Z.
• Where the request was made (the sourceIPAddress). In this case, it is 198.162.198.64.
• The action the request is trying to perform (the eventName). In this case, it is the StartLogging operation.

As a security precaution, you can use events such as this example to configure alerts when an IAM user attempts to sign in to the AWS Management Console too many times.

Global Service Events

Some AWS services allow you to create, modify, and delete resources from any region. These are referred to as global services. Examples of global services include the following:

• IAM
• AWS Security Token Service (AWS STS)
• Amazon CloudFront
• Amazon Route 53

Global services are logged as occurring in US East (N. Virginia) Region. Any trails created in the CloudTrail console log global services by default, which are delivered to the Amazon S3 bucket for the trail.