As discussed in the previous section, you can easily create different CloudWatch metrics and alarms for monitoring your CloudTrail Log files. Luckily for us, AWS provides a really simple and easy to use CloudFormation template, which allows you to get up and running with a few essential alarms in a matter of minutes! The best part of this template is that you can extend the same by adding your own custom alarms and notifications as well. So without any further ado, let's get started with it.
The template itself is fairly simple and easy to work with. You can download a version at https://s3-us-west-2.amazonaws.com/awscloudtrail/cloudwatch-alarms-for-cloudtrail-api-activity/CloudWatch_Alarms_for_CloudTrail_API_Activity.json.
At the time of writing this book, this template supports the creation of metric filters for the following set of AWS resources:
- Amazon EC2 instances
- IAM policies
- Internet gateways
- Network ACLs
- Security groups
- To create and launch this CloudFormation stack, head over to the CloudFormation dashboard by navigating to https://console.aws.amazon.com/cloudformation.
- Next, select the option Create Stack to bring up the CloudFormation template selector page. Paste https://s3-us-west-2.amazonaws.com/awscloudtrail/cloudwatch-alarms-for-cloudtrail-api-activity/CloudWatch_Alarms_for_CloudTrail_API_Activity.json in the Specify an Amazon S3 template URL field, and click on Next to continue.
- In the Specify Details page, provide a suitable Stack name and fill out the following required parameters:
- Email: A valid email address that will receive all SNS notifications. You will have to confirm this email subscription once the template is successfully deployed.
- LogGroupName: The name of the Log Group that we created earlier in this chapter.
- Once the required values are filled in, click on Next to proceed. Review the settings of the template on the Review page and finally select the Create option to complete the process.
The template takes a few minutes to completely finish the creation and configuration of the required alarms. Here is a snapshot of the alarms and metrics that get created for your environment:
|
Logical ID of resources created |
Type of resource |
|
AlarmNotificationTopic |
AWS::SNS::Topic |
|
AuthorizationFailuresAlarm |
AWS::CloudWatch::Alarm |
|
CloudTrailChangesAlarm |
AWS::CloudWatch::Alarm |
|
CloudTrailChangesMetricFilter |
AWS::Logs::MetricFilter |
|
ConsoleSignInFailuresAlarm |
AWS::CloudWatch::Alarm |
|
ConsoleSignInFailuresMetricFilter |
AWS::Logs::MetricFilter |
|
EC2InstanceChangesAlarm |
AWS::CloudWatch::Alarm |
|
EC2InstanceChangesMetricFilter |
AWS::Logs::MetricFilter |
|
EC2LargeInstanceChangesAlarm |
AWS::CloudWatch::Alarm |
|
EC2LargeInstanceChangesMetricFilter |
AWS::Logs::MetricFilter |
|
GatewayChangesAlarm |
AWS::CloudWatch::Alarm |
|
GatewayChangesMetricFilter |
AWS::Logs::MetricFilter |
|
IAMPolicyChangesAlarm |
AWS::CloudWatch::Alarm |
|
IAMPolicyChangesMetricFilter |
AWS::Logs::MetricFilter |
|
NetworkAclChangesAlarm |
AWS::CloudWatch::Alarm |
|
NetworkAclChangesMetricFilter |
AWS::Logs::MetricFilter |
|
SecurityGroupChangesAlarm |
AWS::CloudWatch::Alarm |
|
SecurityGroupChangesMetricFilter |
AWS::Logs::MetricFilter |
|
VpcChangesAlarm |
AWS::CloudWatch::Alarm |
|
VpcChangesMetricFilter |
AWS::Logs::MetricFilter |
So far, we have seen how to integrate CloudTrail's Log files with CloudWatch Log Groups for configuring custom metrics as well as alarms for notifications. But how do you effectively analyze and manage these logs, especially if you have extremely large volumes to deal with? This is exactly what we will be learning about in the next section, along with the help of yet another awesome AWS service called Amazon Elasticsearch!