Brian Beach1, Steven Armentrout2, Rodney Bozo3 and Emmanuel Tsouris4
(1)
Raleigh, NC, USA
(2)
Mountlake Terrace, WA, USA
(3)
Sterling, VA, USA
(4)
North Bend, WA, USA
Two AWS Systems Manager features that help manage your fleet at scale are Systems Manager Inventory and Systems Manager Patch Manager. With Systems Manager Inventory, you can specify the type of metadata you want to collect from your instances, which instances to collect it from, and when to collect it. AWS Systems Manager Patch Manager gives you the tools you need to automate the process of scanning or installing patches on your instances.
In this chapter, we’ll first look at how we can use AWS Systems Manager Inventory to collect some information from our instances. Secondly, we’ll run through the basics of configuring and using AWS Systems Manager Patch Manager to keep our instances up to date on a regular schedule. Finally, we’ll end the chapter with two exercises. The first one walks us through configuration of inventory, and the second one focuses on a common patch management scenario.
Note
AWS Systems Manager Inventory and AWS Systems Manager Patch Manager both build on the concepts discussed in earlier chapters, specifically Systems Manager prerequisites and AWS Systems Manager (SSM) documents. These previous chapters covered an important prerequisite, the IAM instance profile. By default, AWS Systems Manager is not able to do anything with your EC2 Instances; so to enable connectivity between the AWS Systems Manager and the Amazon SSM Agent on your EC2 instances, you’ll need that IAM instance profile discussed in those chapters, with the correct IAM role attached.
AWS Systems Manager Inventory
Systems Manager Inventory gives you the ability to define metadata collection rules. Assuming you’ve attached correct IAM instance profile needed by the Amazon SSM Agent, inventory collection will occur based on the rules and schedule you specify. In the next section, we’ll take a look at how we define those both in the console and using PowerShell.
Once you click Inventory, you’ll be taken to the AWS Systems Manager Inventory Dashboard, where you can set up inventory or look at data. See Figure 17-2.
Figure 17-2
Systems Manager Inventory Dashboard in the console
Creating an Inventory Association
Assuming you have one or more managed instances already running in a particular region, here’s how you can enable Inventory by creating an Inventory Association. We are going to focus on what’s referred to as a global inventory association, and that means that all instances in a particular region will begin collecting inventory data. We can also specify a specific instance, and we’ll see how to do that in the next few steps.
If you’d like to inventory all of your instances in a particular region, you can create a global inventory association by using New-SSMAssociation and specifying a wildcard for the Instance ID.
$target = @{Key = "InstanceIds"; Values = "*"}
We’ll define a schedule using the rate format.
$schExp = "rate(30 minutes)"
Define parameters that tell SSM what items to inventory (applications, AWS components, etc.).
$params = @{
applications='Enabled';
awsComponents='Enabled';
customInventory='Enabled';
instanceDetailedInformation='Enabled';
networkConfig='Enabled';
services='Enabled';
windowsRoles='Enabled';
windowsUpdates='Enabled'
}
Then, we create an association for the AWS-GatherSoftwareInventory document, passing the target, schedule, and parameters.
In order to understand what inventory types are available, we can look at the schemas available to us in a particular region using Get-SSMInventorySchema.
Get-SSMInventorySchema -Region us-west-2
If we want to look at just the type names available in a nice list, we can simply look at the TypeName property.
To view the inventory schemas that support the use of an aggregator, we just pass $true to the -Aggregator parameter. An aggregator groups and summarizes larger amounts of inventory data.
Looking at the inventory result entity data structure, we can see two properties. There’s an ID which corresponds to the Instance ID, and a data property.
PS C:> $inventoryResultEntity | Format-List
Data : {[AWS:InstanceInformation, Amazon.SimpleSystemsManagement.Model.InventoryResultItem]}
Id : i-1234567890
The Data property contains a key value pair. We can look at this property to see the specific inventory schema data type (key) and inventory data (value). You might see different data types and values depending on the specific inventory data that has been collected in your account.
PS C:>$inventoryResultEntity.Data | Format-List
Key : AWS:InstanceInformation
Value : Amazon.SimpleSystemsManagement.Model.InventoryResultItem
Now let’s look at the content stored within the value property.
When we look at $inventoryContent, we can see various key value pairs which contain our inventory data. For example, the AWS:InstanceInformation data type gives us AgentType, AgentVersion, InstanceStatus, IpAddress, PlatformName, PlatformType, and other relevant properties related to an Amazon EC2 Instance.
To look at a specific data type, for a specific instance, we can use Get-SSMInventoryEntryList.
We can see the results by looking at the Entries property.
PS C:>$instInv.Entries
Key Value
--- -----
Description Update
HotFixId KB4091664
InstalledBy NT AUTHORITYSYSTEM
InstalledTime 2019-01-01T00:00:00Z
Aggregating Inventory Data
Oftentimes, you’ll have a lot of inventory data and need to look at a summary (or aggregate) of that data. We first need to create an InventoryAggregator object.
Let’s look at the values property of the object we get back.
$invAggResultItem = ($invAggResults.Data).Values
Now, look at the content property. If you have multiple Windows and Linux instances that have run inventory, you’ll see a count of those.
PS C:>$invAggResultItem.Content
Key Value
--- -----
Count 2
PlatformType Linux
Count 18
PlatformType Windows
AWS Systems Manager Patch Manager
Keeping your running instances up to date can be a challenge, especially at scale. Having a systems management solution at your fingertips can not only save you time, but keep your infrastructure and applications secure. AWS Systems Manager Patch Manager helps you automate the actions you’d normally have to take on your Windows and Linux systems to keep them up to date.
Before we begin, we’ll dive a little deeper into some of the basic concepts of Patch Manager: patch baselines, patch groups, and maintenance windows.
Patch Baselines
There are two sorts of patch baselines with AWS Systems Manager, one is the default baseline, and the other is a custom patch baseline. With the default baseline, you get a set of predefined patch baselines for various operating systems. A custom patch baseline is one that you define, and tell Patch Manager how and what to patch. This can give you the granularity, for example, to make sure a certain operating system patch doesn’t take down your application.
Viewing Existing Patch Baselines
To view the patch baselines available to us in a given region, we can use Get-SSMPatchBaseline, and it will return a list of our baselines in addition to the default ones provided by AWS.
Get-SSMPatchBaseline -Region $region
In the list, we’ll see a number of patch baselines, including the default ones for Windows and Amazon Linux.
BaselineDescription : Default Patch Baseline Provided by AWS.
To create a new custom patch baseline, we’ll need to set up a few variable first to make things a little easier. First, we’ll set up a name, description, and tags for our new patch baseline.
$pbName = "Development-Baseline"
$pbDesc = "My Patch Baseline"
$pbTags = @{Key="Environment";Value="Production"}
Now, we’ll need a couple of patch filters which consist of key/value pairs. For a list of valid options, take a look at the class reference for Amazon.SimpleSystemsManagement.Model.PatchFilter in the AWS SDK for .Net.
To remove (or delete) a patch baseline, we simply use the Remove-SSMPatchBaseline cmdlet and pass it a valid baseline ID. If you want to remove the patch baseline without PowerShell prompting you to confirm, simply add the -Force switch.
When patching your servers, you may find yourself needing a way to group your instances by some arbitrary grouping strategy.
Some common groupings might include
Production, Testing, and Development Server Groups
Frontend and Backend Server Groups
Web and Database Server Groups
Windows and Linux Server Groups
Patch groups are simply created with Amazon EC2 tags. If you have spent a lot of time tagging your instances into some logical grouping, this capability gives you the power to use those tags for your patch and update management needs.
Viewing Patch Groups
To see patch groups (or a specific patch group), we use the Get-SSMPatchGroup cmdlet.
Get-SSMPatchGroup -Region $region
If we are looking for a high-level aggregated patch compliance state for a patch group, then we can use the Get-SSMPatchGroupState cmdlet.
For registering (or associating) a patch baseline to a patch group, we can use the Register-SSMPatchBaselineForPatchGroup cmdlet and pass it the baseline ID, along with a patch group name.
AWS Systems Manager Patch Manager uses maintenance windows to define when software updates should be applied to instances. This allows us to pick a reasonable schedule for any activities which might impact users or business processes. Maintenance windows are defined with a schedule, a maximum duration, registered targets (instances), and tasks. There are four distinct tasks we can use with maintenance windows: Run Command, AWS Systems Manager Automation workflows, AWS Lambda functions, and tasks within AWS Step Functions.
Viewing Maintenance Windows
Listing all of the maintenance windows in a given account for a specific region is done using the Get-SSMMaintenanceWindowList cmdlet.
Get-SSMMaintenanceWindowList -Region $region
Getting information for an existing maintenance window is done using the Get-SSMMaintenanceWindow cmdlet and passing it an existing maintenance window ID.
When we want to create a maintenance window, we can do so by using the New-SSMMaintenanceWindow cmdlet and passing a cron or rate expression as the schedule, duration in hours, and cutoff which defines the number of hours before the end of a window when SSM should stop scheduling new tasks.
In order for instances to be associated with a maintenance window, we must register them using the Register-SSMTargetWithMaintenanceWindow cmdlet and pass a maintenance window ID, target, owner information, and resource type.
Here, we’ll define a task target using a maintenance window target ID.
Oftentimes, you’ll have instances running in your AWS account and will need to know what’s on those instances. In this exercise, we’ll look at enabling inventory for a single Amazon EC2 instance. Once inventory has been collected, we’ll look for some applications and settings using the inventory data.
Do We Have Managed Instances Running?
Before we can configure inventory, we’ll need at least one (or more) Amazon EC2 instances. The instances must have an instance profile attached with the correct role. This enables AWS Systems Manager (SSM) to communicate with the agent running on the instance. If you’d prefer to use an EC2 instance you already have running, feel free to do so, but just make sure it has the necessary instance profile attached.
Launch a Windows Instance with SSM Enabled
To begin this exercise, we’ll first define the region we’ll be working in.
$region = "us-west-2"
Next, let’s define the AWS Systems Manager Parameter Store path which will give us the latest Windows Server 2016 AMI.
Now we’ll define a cron or rate expression for the inventory collection schedule. We’ll use 30 minutes here, but in production work, you’d want to spread the inventory schedule out a little more.
$schExp = “rate(30 minutes)”
Let’s enable a few inventory items.
$params = @{
applications='Enabled';
awsComponents='Enabled';
customInventory='Enabled';
instanceDetailedInformation='Enabled';
networkConfig='Enabled';
services='Enabled';
windowsRoles='Enabled';
windowsUpdates='Enabled'
}
Now we’ll associate the AWS Systems Manager document, AWS-GatherSoftwareInventory, to our running instance using the parameters we just set.
Since we configured it to run inventory every 30 minutes, we’ll need to wait at least that long before we can see any inventory data for this instance.
Checking Inventory Status
We can look at the SSM associations which will include the one we created for inventory collection.
Get-SSMAssociationList -Region $region
By passing the association ID of our association, we can view the status of it.
You can see now that when inventory is complete, we have some interesting data which can help us manage our instances. Many features within AWS Systems Manager work together to give you a complete systems management solution. Inventory, just like many of the other Systems Manager features, uses documents and associations, one of the foundational concepts of how SSM works.
Exercise 17.2: Patching During a Maintenance Window
Let’s take a look at how we would patch servers using a custom patch baseline, some patch groups, and a maintenance window. Since Microsoft releases patches the second Tuesday of the month, we’ll create two maintenance windows. One will be a recurring Wednesday night scan task, and the other will be a patching task on every Friday night.
Create a Custom Patch Baseline
First, we’ll begin by defining our region.
$region = "us-west-2"
Next, let’s create a name and description for our patch baseline.
$pbName = "Production-Patch-Baseline"
$pbDesc = "My Patch Baseline"
We’ll give our patch baseline a production tag, so we know which environment we’ve created it for.
$pbTags = @{Key="Environment";Value="Production"}
Now, we’ll need a patch filter to define what patches this will be applicable to. Let’s set the first filter to only include Critical and Important patches (as rated by the Microsoft Security Response Center), represented by MSRC_SEVERITY.
Our new patch group then gets added to a patch rule. The rule will specify how many days should pass before approving the patch and also specify the compliance level.
Let’s create a tag that will let us know our maintenance windows are for our production environment.
$tags = @{Key="Environment";Value="Production"}
Now, we can create our Wednesday morning maintenance window by creating a cron expression for 9 p.m. every Wednesday. The duration of the maintenance window will be 2 hours after it begins, and we’ll set the cutoff to 0.
We’ll do it again, but this time we’re creating a Friday night maintenance window. Since this will be our patching Window, let’s give it 3 hours with a cutoff of 1 hour. The cutoff prevents any new tasks from starting after 11 p.m.
For our scan task, we’ll run a scan on our production servers every Wednesday night. Let’s define a task target by creating a hashtable where WindowTargetIds is the key and our target registered to our Maintenance Window for Wednesday is the value.
Since we specified when we created our maintenance window, we’ll need our EC2 instances to have a patch group tag. Let’s tag an instance with Patch Group as the key and Production (our patch group name) as the value.
Once our scanning and patch maintenance windows have completed, we’ll be able to look at the state of our patch groups and also be able to tie it into inventory data for a more complete picture of our servers.
Summary
In this chapter, we learned about AWS Systems Manager Inventory and Patch Manager, two features you can use to manage your fleet of instances. We went through some steps to configure inventory and query that data. We also took a deep look at patch management and maintenance windows. Using PowerShell, you can automate some of the more complex tasks related to systems management and manage your servers in the cloud. This information coupled with the other chapters on AWS Systems Manager gives you a starting point to building a cloud-based systems management solution for your specific needs.
