Brian Beach1, Steven Armentrout2, Rodney Bozo3 and Emmanuel Tsouris4
(1)
Raleigh, NC, USA
(2)
Mountlake Terrace, WA, USA
(3)
Sterling, VA, USA
(4)
North Bend, WA, USA
Now that you’ve been creating resources in the cloud with PowerShell, you’ll need a simple and secure way to manage all those resources. AWS Systems Manager is a convenient set of management tools that lets you configure and manage your AWS resources at scale.
In this chapter, I’ll explain some basics to get you started with Systems Manager. We’ll begin by going over some important prerequisites, and then I’ll show you how to get organized by grouping your resources using AWS Resource Groups. We’ll walk through Session Manager which allows you to connect and run PowerShell on your Amazon EC2 instances. Finally, I’ll show you a few ways to use Parameter Store with your PowerShell scripts.
There are two exercises in this chapter. In the first, I’ll show you how to use Parameter Store to find the latest Windows AMI and use it to launch an EC2 instance. In the second exercise, you’ll create a resource group to list resources that match a certain resource query.
To explore and learn more about the many features we’ll be talking about, open your browser and head over to the AWS Systems Manager Console at https://console.aws.amazon.com/systems-manager.
Note
If you’ve been using AWS for a while, you’ll remember AWS Systems Manager under its former name: Amazon EC2 Systems Manager.
Systems Manager Prerequisites
Remember when we covered IAM in detail back in Chapter 11? Well before you begin using Systems Manager, there are a few IAM permissions you’ll need to set up.
IAM Access to Systems Manager
If your IAM user or role has full administrative access to your AWS account, you won’t need to add any additional permissions. For those of you who need to configure your IAM user with the least amount of privileges, there are a few permissions you’ll need to add depending on the specific features you’ll be using. You can find the details at https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-access-user.html.
Creating an IAM Instance Profile for Systems Manager
Systems Manager won’t be able to do anything with your Amazon EC2 instances by default. The way you allow Systems Manager to take action, connect, or manage your instances is with an IAM instance profile. You can create one or many roles with the specific permissions you need. We talked about IAM and instance profiles back in Chapters 2 and 3. We’ll cover it again in the next few steps, focusing on the permissions needed for Systems Manager.
We’ll first define a policy that will allow the Systems Manager service to assume a role. Once we have that policy in JSON format, we’ll create the new role and attach the policy. Then we must create a new instance profile and add the IAM role to that instance profile.
Next, we create the new IAM role with the assume role policy JSON. The assume role policy allows the Systems Manager service (ssm.amazonaws.com) to assume the new role we are creating.
Now, attach the AmazonEC2RoleforSSM managed policy to the role using Register-IAMRolePolicy, and pass in the ARN for the managed policy. This managed policy permits communication to the Systems Manager API.
The Systems Manager profile in this example doesn’t allow access to other services such as Amazon S3. Be sure to create instance profiles that have access to only the services needed in each specific use case.
To use this profile with Systems Manager, you’ll need to either launch a new instance with the profile attached as discussed in Chapter 3, or attach it to an existing EC2 instance.
AWS Resource Groups
Now that you’ve been creating resources, you need a way to group them and keep things organized. With AWS Systems Manager Resource Groups, you’re able to group your many AWS resources by tags. You can use Resource Groups in many different ways that will help you manage your cloud resources at scale. Some examples for grouping your resources are to group them by application, by environment (development, test, production), or for billing (by customer, department, or cost center).
Note
Systems Manager Resource Groups are regional, so you’ll only see AWS resources in the same region. To view resources in other regions, you’ll need to create a resource group in those regions as well.
Creating New Resource Groups
In order to create a new resource group, we’ll need a JSON query which represents the tags by which we’ll group resources. I’ve built the JSON using PowerShell hashtables, but you can just as easily pass the JSON from a file or string.
First, let’s begin by defining a hashtable for the tag we’ll be filtering on.
$tagFilter = @{}
$tagFilter["Key"] = "Application"
$tagFilter["Values"] = @("MyApplication")
Next, we’ll define another hashtable that represents the query and we’ll reference our tag filter hashtable. In this query we’re limiting the resources to EC2 instances and S3 buckets. These resource filters follow the standard naming scheme found in AWS ARNs. You can find a complete list of supported resources at https://docs.aws.amazon.com/ARG/latest/userguide/supported-resources.html.
Now that we have our query constructed, we can convert it to JSON. We need to make sure we have the depth set with ConvertTo-Json, and we’ll also use the compress switch to remove whitespace.
If you receive an invalid operation exception in PowerShell telling you that it is unable to find [Amazon.ResourceGroups.QueryType], then you are most likely using an older version of the AWS PowerShell module which is missing the object we are trying to create. The way to fix this error is to update your AWS PowerShell module.
Now it’s time to put it all together into a resource query object. We’ll do that by using New-Object and assigning $queryType and $queryJSON to the type and query properties, respectively.
Finally, we can create our resource group using New-RGGroup and passing our resource query object.
New-RGGroup -DescNew-RGription "My Resource Group Description" -Name "MyFirstResourceGroup" -ResourceQuery $resourceQuery -Region us-east-1
Updating Resource Groups
Let’s take a look at how to update a resource groups. In the previous section, we constructed a query with hashtables and converted the hashtables to JSON. An alternative method is to pass the query as a string already formatted as JSON.
We’re going to pass the query as a JSON-formatted string. In the previous example, we passed EC2 instances and S3 buckets as the resource type filters; this time let’s limit to just EC2 instances.
Once we have our new resource query object, we can update the resource group query by passing the resource query and resource group name to Update-RGGroupQuery.
There may be times when you want to test a resource query or quickly search for resources without first creating a resource group. Using Find-RGResource, you can do just that. Just as we did before, we’ll define a query type for tag filters.
Finally, we’ll pass the resource query object to Find-RGResource.
Find-RGResource -ResourceQuery $resourceQuery
-Region us-east-1
Listing Resource Groups
To list resource groups in your account, use Get-RGGroupList which returns the name, description, and ARN of the groups.
Get-RGGroupList -Region us-east-1
Deleting Resource Groups
Use Remove-RGGroup to delete groups. PowerShell will ask you to confirm the action to be sure you really want to delete the group. You can use the –Force switch to override the confirmation.
Performing the operation "Remove-RGGroup (DeleteGroup)" on target "MyFirstResourceGroup".
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "Yes"): Y
Description GroupArn Name
----------- -------- ----
Test Description arn:aws:reso... MyFirstResourceGroup
Listing Resources in a Resource Group
There are a few ways to view resources in resource groups. With the Systems Manager Console, you get a rich visual view of your resources. For scripting and automation, you’ll want to use Get-RGGroupResourceList and pass the group name.
Within the Systems Manager Console, under the “Resource Groups” section, you’ll find Insights. We’ll go into detail on some of these in the next chapter, but there’s one feature here that directly ties into the Resource Groups we just learned about. That’s the Built-In Insights as shown in Figure 15-1. From the Built-In Insights, you first choose a resource group and can then view AWS Config rule compliance, Resource compliance, and Config history. This is a powerful feature to keep track of your resources and the changes made to them.
Figure 15-1
The Built-In Insights within AWS Systems Manager Console
Session Manager
Using Session Manager, you’re able to connect to your Windows and Linux EC2 instances with a powerful web-based interactive shell. We’ll take a look at Session Manager with a Windows instance, which gives us a PowerShell command-line interface in a web browser!
Tip
Session Manager gives you command-line access to your instances. If you typically use RDP or SSH to run command and scripts, you may be able to keep those inbound ports closed and just use Session Manager instead!
The AWS Systems Manager Console, where Session Manager can be found
Figure 15-3 shows you the Preferences tab. You have the option to set an S3 bucket for storing output, configure encryption, and enable CloudWatch logging.
Figure 15-3
The Session Manager Preferences tab allows you to configure a location for storing output and enabling CloudWatch Logs
Back on the Sessions tab in Figure 15-4, you’ll see a list of sessions and their status.
Figure 15-4
The Sessions tab shows you a list of current sessions
On the upper right of the Session Manager Console, you’ll see the Start session button as shown in Figure 15-5. Click the button and select an instance to connect to.
Figure 15-5
The Start session button allows you to begin a new session
Once connected, as in Figure 15-6, you’ll have a regular PowerShell session in your web browser. From here, you can run any of the usual commands you would run on an instance. Two great aspects of this feature are that you don’t need remote desktop and you don’t need to open port 3389 on your security group.
Figure 15-6
A PowerShell session from your instance in a web browser
Note
Connecting to an instance with Session Manager requires that you have the Systems Manager agent installed and running on your instance. The agent is preinstalled on Amazon-provided AMIs by default. You’ll also need to launch your instance with an instance profile that has Systems Manager permissions associated with it. See the previous section in this chapter on “Creating a Systems Manager IAM instance profile.”
Parameter Store
Parameter Store is a powerful shared feature within AWS Systems Manager and is designed to store configuration parameters for anything you build in the cloud. People often use it for application settings, database connection strings, passwords, license keys, and other runtime parameters. Parameter Store also features public parameters that can help you find the latest Windows or Linux AMI. You’ll find Parameter Store at the bottom of the AWS Systems Manager Console, under Shared Resources as shown in Figure 15-7.
Figure 15-7
Parameter Store can be found under the Shared Resources menu
Using Parameter Store
A parameter’s name follows a hierarchical format. This allows you to get all parameters matching the hierarchical path. You create entries in Parameter Store with Write-SSMParameter, though keep in mind as with most AWS resources, your parameters are stored in the region you create them.
Now, let’s look at using that hierarchical path to get any parameters that begin with /Test. This returns a parameter object where the value we set is stored in the value property.
Finally, to remove a parameter, you call Remove-SSMParameter. PowerShell will prompt you to confirm. If you would like to suppress the confirmation prompt, remember to use the –Force switch.
One of the Parameter Store features I use the most is finding the latest Windows AMI published by Amazon using a public parameter. Let’s take a look at how this works using Get-SSMParametersByPath, which will return all the entries with the path /aws/service/ami-windows-latest, and the respective AMI Ids.
From the list returned by Get-SSMParametersByPath, we can pick a specific parameter and use Get-SSMParameter to return the AMI ID for that specific entry.
We saw how we were able to retrieve the latest Windows AMI using Get-SSMParametersByPath, we can do the same for Amazon Linux AMIs which are under the path /aws/service/ami-amazon-linux-latest:
Systems Manager has quite a few capabilities such as Run Command, Automation, and State Manager where you can reference Parameter Store values by using a specific syntax of {{ssm:ParameterName}}.
For a parameter with the name of /Test/ParameterName, you would use the following syntax:
{{ssm:/Test/ParameterName }}
We’ll dive deeper into Run Command, Automation, and State Manager in the next couple of chapters, so remember that syntax for Parameter Store.
Exercise 15.1: Launch An EC2 Instance With The Latest AMI
When running Windows, it’s important to use the latest AMI which contains the latest Microsoft Updates along with the latest drivers for Amazon EC2. If you’ve ever run Get-EC2Image without specifying a filter or specific image ID for an AMI, you’ll know that there are quite a few AMIs available! If you’re new to AWS, it can indeed be difficult to determine which AMI to use when launching a new Amazon EC2 Instance.
In this exercise, we’ll be using Parameter Store to quickly and easily find the latest Windows AMI and launch a new instance using it. In order to keep track of this new instance, we’ll also use what we learned about tagging an EC2 instance in earlier chapters to apply a couple of tags.
Write Some Reusable Values to Parameter Store
First, let’s create a variable to use for our region. Then we’ll store a few values in Parameter Store to reuse in our script.
$region = "us-east-1"
The first value will be the name of our application.
The second value is what we’ll use to set the name of our application server instance.
Write-SSMParameter -Name "/Test/Application/ServerName" -Description "The application server name." -Value "MyTestApplicationServer" -Type String -Region $region
Third, we’ll need to set the instance type to use when spinning up an EC2 instance for the application server. We’ll set this in Parameter Store, so when we need to scale up the instance type in future launches, we can do so without needing to modify the script.
Write-SSMParameter -Name "/Test/Application/ServerInstanceType" -Description "The application server instance type." -Value "t3.large" -Type String -Region $region
Get the Parameter Store Values
Here, we’re going to grab the application server name, the application name, and instance type from Parameter Store.
In this step, we’ll create a tag specification using New-Object and reference the tag specification type. Then we’ll set the resource type to instance, since this is being used with an EC2 instance.
Next, we’ll use the parameter path we just defined with Get-SSMParameter and store the result. The object returned has a few different properties and we only need the image ID for the AMI in the next step, so we’ll use the value property.
Finally, we can assign the RunningInstances property of the newly created instance to a variable.
$runningInstance = $newInstance.RunningInstance
If we look at this $runningInstance variable, we will see details for the running instance. If we wanted to build more automation around our running instance, we could use this variable to pass the Instance ID to another function. For now, we’ll keep it simple and stop here.
Now we have seen how easy it is to create Parameter Store values and use them to launch an EC2 Instance. Our instance has two tags, the first one is a name that shows up in the main EC2 Console view, and the second is an application tag that we’ll use in the next exercise.
Exercise 15.2: Creating And Using a Resource Group
When building in the cloud with all the different AWS services, we need an easy way to group all of our resources together. Luckily, we can use Resource Groups to do just that!
Creating a Resource Group
Begin by creating a region variable and retrieving the application name from Parameter Store. Remember, we wrote to Parameter Store in Exercise 15.1.
Finally, we’re going to create the resource group using the same application name to prepend the name of the resource group.
New-RGGroup -Name "$applicationName -ResourceGroup" -Description "My Resource Group Description" -ResourceQuery $resourceQuery -Region $region
Listing Resources in a Resource Group
In this next activity, we are going to use the resource group we just created to list what resources are picked up by the query. First, we’ll create a variable for our region.
$region = "us-east-1"
Then we’ll grab our application name from Parameter Store.
In this exercise you learned how to use Resource Groups to group resources for an application using a common tag. To better explore how resource groups can help you manage your resources, you can try tagging an Amazon S3 bucket with the same tag as the resource group query and see if it shows up in the resource group.
Summary
We saw that Resource Groups are a powerful way to group your AWS resources by using queries based on tags. With Built-In Insights, we saw how we can use our Resource Groups to look at some useful insights that are built right into the Systems Manager Console. Taking a look at Session Manager, we learned how we can run PowerShell from a web-based console without the need for RDP to our Windows instance. We wrapped up the chapter with Parameter Store and learned how to find the latest Windows or Linux AMIs, store custom application settings, and use those settings in our script. These are just a few of Systems Manager’s powerful tools to help you manage your fleet of resources.
In the next chapter, we’ll continue to dive into Systems Manager, occasionally coming back to both Parameter Store and Resource Groups as we discuss the different features that integrate with them.
