© Brian Beach, Steven Armentrout, Rodney Bozo, Emmanuel Tsouris 2019
B. Beach et al.Pro PowerShell for Amazon Web Serviceshttps://doi.org/10.1007/978-1-4842-4850-8_7

7. Amazon Machine Images

Brian Beach1 , Steven Armentrout2, Rodney Bozo3 and Emmanuel Tsouris4
(1)
Raleigh, NC, USA
(2)
Mountlake Terrace, WA, USA
(3)
Sterling, VA, USA
(4)
North Bend, WA, USA
 

In the last few chapters, we have focused on creating and managing instances. This chapter is about the templates we use to create those instances. Amazon refers to these templates as Amazon Machine Images (AMIs). In this chapter we will explore the AMIs that already exist, and we will discuss how to create your own AMI and share it with others. Finally, we learn how to import a VM from VMware or Hyper-V into AWS.

Many users will never have occasion to create a custom AMI. Most users will be happy with the countless images that Amazon and its partners make available. But some users will want to have complete control over their environment. For example, you may have a corporate server image that you want to make available to your companies’ employees that are using AWS.

As your experience progresses, you will likely find that you want to automate instance builds. Configuration management tools are all about scripting server builds to minimize build time and ensure consistency between builds. Assuming you want to automate the build, there are many options. Most fall on a spectrum somewhere between scripted builds and prepared images.

Working with Scripted Builds and Prepared Images

At one end of the spectrum is the scripted build. With a scripted build, you start with a generic image and use a series of scripts to configure the server as needed. For example, to create a Web Server, you might start with the Amazon Windows Server 2019 Base image. Then you could use the user data to include a custom PowerShell script that enables the Web Server role and downloads the application from source control.

At the other end of the spectrum is the prepared image. With a prepared image, you configure the server, usually manually, and then create an image. When a user needs a new server, he or she selects your server image and creates a new instance. If you choose a prepared image, be sure to update the image periodically with the latest security patches and virus definitions.

Both options have benefits and drawbacks. The scripted build is best when the application is changing often. You always get the latest code and can change the script as requirements change. The prepared image, on the other hand, is best when the application is stable. There are fewer external dependencies that can cause errors and the build is usually faster.

Of course, there are many options on the spectrum between scripted build and prepared image. The Amazon Windows AMIs provide a good example. Amazon offers a base image as well as SQL Server images. By using the SQL Server image, you do not have to script the configuration of SQL Server. You simply focus on scripting the deployment of your application.

Most of this chapter is focused on preparing images, but don’t overlook scripting as an option. There are many AWS Services that will help you script instance configuration such as CloudFormation and OpsWorks. Chapters 15–17 will cover Systems Manager which can be used to script instance configuration with Ansible, Salt, or PowerShell Desired State Configuration (DSC) .

Listing AMIs

Before we create our own AMI, or simply an image, let’s take a deeper look at the images that are already available. We don’t want to spend time creating and maintaining an image if an identical image already exists.

Caution

There are over 100,000 images available giving you a ton of options to choose from, but be careful! You should only launch images from publishers you trust. As you will see later in this chapter, anyone can publish an image.

You can find images using the Get-EC2Image command, but this command will return the complete list of over 100,000 images. Obviously, this is far too many to look through one at a time.

Limiting the Number of Instance Results

As you might expect, you can use filters to limit the number of instances. For example, if you are interested in a Windows image, you can use the platform filter. The following example will return about 15,000 Windows images in the Northern Virginia region.
$Filter = New-Object Amazon.EC2.Model.Filter
$Filter.Name = "platform"
$Filter.Value = "windows"
Get-EC2Image -Filter $Filter | Select-Object Name
We can also filter by publisher using owner-alias. For example, you might list only those images that Amazon publishes. Again, it is a really good idea to only use images published by an owner you trust, such as Amazon. The following example will return about 5000 images:
$Filter = New-Object Amazon.EC2.Model.Filter
$Filter.Name = "owner-alias"
$Filter.Value = "amazon"
Get-EC2Image -Filter $Filter | Select-Object Name
This is still too many images to comb through one by one. Of course, you can combine two or more filters. If we combine the platform and owner alias, we get a more reasonable list of about 1800 images.
$Filter1 = New-Object Amazon.EC2.Model.Filter
$Filter1.Name = "platform"
$Filter1.Value = "windows"
$Filter2 = New-Object Amazon.EC2.Model.Filter
$Filter2.Name = "owner-alias"
$Filter2.Value = "amazon"
Get-EC2Image -Filter $Filter1, $Filter2 | Select-Object Name

Finding an Instance by Name

The prior examples assume you do not yet know which image you are looking for. If you know the name of the image you want to find, you can use the name filter . For example, to find the Windows Server 2019 Base image, use
$Filter = New-Object Amazon.EC2.Model.Filter
$Filter.Name = "name"
$Filter.Value = "Windows_Server-2019-English-Full-Base-2019.01.10"
Get-EC2Image -Filter $Filter
Note that that command is going to fail. Amazon updates most of its images periodically with the latest patches and updates. By the time you read this, that image will likely no longer exist. Luckily, filters support the wildcard character (*). For example:
$Filter = New-Object Amazon.EC2.Model.Filter
$Filter.Name = "name"
$Filter.Value = "Windows_Server-2019-English-Full-Base*"
Get-EC2Image -Filter $Filter

So, let’s review. We can use a combination of the platform and owner-alias filters to discover new images from a trusted source. Then, once we know the name, we can search by name. If all of this seems cumbersome to you, I agree. Wouldn’t it be great if we had a short list of the most common images?

Locating the Most Common Images

Luckily Amazon thought of the idea of getting a short list of the most common images and included another command, Get-EC2ImageByName . This command will return most of the images that you find on the Quick Start tab of the New Instance Wizard in the AWS Management Console. Note that the command may return an array with multiple versions of a given instance. The most recent version will be listed first in the array. For example:
Get-EC2ImageByName -Name "WINDOWS_2016_BASE"

You can run Get-EC2ImageByName without any parameters to get a list of available names.

Finally, if you have launched your own images, as described later in this chapter, you can find them by using the Owner parameter of the Get-EC2Image command. For example:
Get-EC2Image -Owner self

Now that we know how to find images, we can decide whether we need to create our own. Let’s assume that none of the existing images meet our needs and we have decided to create our own image. Images are created using SysPrep and the EC2Config Service. Before we get started creating an image, let’s look at EC2Launch.

Introducing EC2Launch

Before we move on to creating an image, I want to introduce EC2Launch. Note that EC2Launch runs on Windows 2016 or newer images. It replaces the EC2Config Service that was installed on Windows images through 2012 R2. We have mentioned these tools a few times in prior chapters, but now is a good time to look at it in detail.

EC2Launch is used to configure Windows instances. It plays a critical role in configuring an instance when it boots for the first time. For example, the EC2Launch is responsible for encrypting the administrator password and executing scripts in the user data.

When an instance boots for the first time, the EC2Config Service performs the following tasks:
  1. 1.

    Renames the computer. This is disabled by default.

     
  2. 2.

    Sets the administrator password. By default, a new, random password will be generated and encrypted with the specified key pair.

     
  3. 3.

    Creates RDP certificate. A new self-signed host certificate is created for Remote Desktop connection. You cannot use RDP without a certificate.

     
  4. 4.

    Extends the OS partition. Remember that you can change the size of the OS volume at launch. Therefore, the service extends the partition to fill the volume.

     
  5. 5.

    Activates Windows if necessary.

     
  6. 6.

    Writes event log entries to the AWS System Log. This can help debug errors that occur before RDP is available in the boot sequence.

     
  7. 7.

    Creates a new wallpaper image. This includes useful information (name, type, memory, etc.) about the image.

     
  8. 8.

    Configures a few custom routes. For example, 169.254.169.250 and 169.254.169.251 are the default KMS servers and 169.254.169.254 is the metadata URL we used in Chapter 3.

     
Most of these actions are enabled by default, but you can customize them as needed by editing the file:
C:ProgramDataAmazonEC2-WindowsLaunchConfigLaunchConfig.json
There is also a GUI tool available for editing the JSON. It is located at
C:ProgramDataAmazonEC2-WindowsLaunchSettingsEc2LaunchSettings.exe
The GUI looks like Figure 7-1.
../images/319650_2_En_7_Chapter/319650_2_En_7_Fig1_HTML.jpg
Figure 7-1

EC2 Launch Settings

The default behavior is to run EC2Launch on the first launch. However, there may be times when you want to run it on every boot. If you want to configure it, run on every boot, run the following command, or check the check box seen at the bottom of Figure 7-1.
C:ProgramDataAmazonEC2-WindowsLaunchScriptsInitializeInstance.ps1 -SchedulePerBoot

Now that we understand EC2Launch, let’s look at the EC2LaunchSettings tool.

Preparing an AMI Using EC2LaunchSettings

In the prior section, we learned about EC2Launch. In this section we will prepare an image of our own. To start, launch a new Windows Server 2019 Base instance that will serve as our template. You remember how to do that right?

Once the instance boots, you can log in and make whatever changes you want. Let’s assume we are developing a web application and we want to create a server to test it on. Our application requires that we enable a few unique features of IIS.

In Figure 7-2 I have configured the required roles and services. First, I enabled the Web Server (IIS) role (see the Select server roles dialog in Figure 7-2). Next, I enabled the Windows Identity Foundation feature (see the Select features dialog in Figure 7-2). Finally, I enabled six of the nine Web Server Security Role Services (see the Select role services dialog in Figure 7-2). Obviously your configuration will depend on the applications you intend to run on the instance.
../images/319650_2_En_7_Chapter/319650_2_En_7_Fig2_HTML.jpg
Figure 7-2

Configuring the Web Server role

Once you have configured your server and installed any software you want in the template, it is time to prepare the image. As I mentioned in the prior section, you use EC2LaunchSettings to create an image. Behind the scenes, EC2LaunchSettings uses SysPrep to do the heavy lifting.

Caution

Before continuing, you should take a snapshot of the instance. Once we SysPrep the image, there is no going back. If the instance fails to boot, you will have to start over from scratch.

Open the Ec2LaunchSettings application (see Figure 7-1 for reference). Leave the defaults and simply click the Shutdown with SysPrep button. This will take a few minutes. Once it’s done, we can finally create the AMI.

Creating an AMI

The instance is now configured and waiting to run setup. We want to clone the instance in this state, so that each copy runs setup when it first boots. It’s finally time to create an image. Let’s look at the AWS Administration Console first and then discuss the PowerShell commands.

In the AWS Management Console, right-click the instance you want to create an image of and select Create Image (EBS AMI). Figure 7-3 shows the Create Image dialog box. From here you can give your image a name and description and configure the volumes. Remember, from Chapter 4, that the user will have the option of modifying the volume configuration when he or she launches an instance of your image. If the image requires multiple volumes, you can set default values here.
../images/319650_2_En_7_Chapter/319650_2_En_7_Fig3_HTML.jpg
Figure 7-3

Create Image dialog box

The equivalent PowerShell command is New-EC2Image. The command takes the ID of the instance you want to make a template from, as well as a name and description, and returns the ID of the new image. For example:
$AMIID = New-EC2Image -InstanceId i-99999999 -Name "WIN2019WEB" -Description "Windows Web Server"
Note that you must replace the instance ID with the instance you want to create an image of. As you might expect, it takes a few minutes to create the image. You can check the ImageState to see if the image is ready. To wait for an image, you can use a while loop similar to the following example:
$AMI = Get-EC2Image $AMIID
While($AMI.ImageState -ne "available") {
        $AMI = Get-EC2Image $AMIID
        Start-Sleep -Seconds 15
}
Modifying the drive configuration works just like it did when we used the New-EC2Instance command in Chapter 4. Let’s add another 100GB volume to our image to store IIS log files. Remember that the EC2Config Service is configured to automatically mount and format any additional volumes that we attach. All we need to do is create a block device and mapping descriptor and pass it to the New-EC2Image command using the BlockDeviceMapping attribute . For example:
$Volume1 = New-Object Amazon.EC2.Model.EbsBlockDevice
$Volume1.DeleteOnTermination = $True
$Volume1.VolumeSize = 30
$Volume1.VolumeType = "standard"
$Mapping1 = New-Object Amazon.EC2.Model.BlockDeviceMapping
$Mapping1.DeviceName = "/dev/sda1"
$Mapping1.Ebs = $Volume1
$Volume2 = New-Object Amazon.EC2.Model.EbsBlockDevice
$Volume2.DeleteOnTermination = $False
$Volume2.VolumeSize = 100
$Volume2.VolumeType = "standard"
$Mapping2 = New-Object Amazon.EC2.Model.BlockDeviceMapping
$Mapping2.DeviceName = "xvdf"
$Mapping2.Ebs = $Volume2
$AMIID = New-EC2Image -InstanceId i-9999999
     -Name "WIN2019WEB2" -Description "Windows Web Server 2"
     -BlockDeviceMapping $Mapping1, $Mapping2

At this point you have your own custom AMI and you can create instances. This same process can be used to make as many variations as you need. If you find that an image is particularly useful, you may want to share it with others. In the next section, I will show you how to share your image.

Sharing an AMI

You may find that you want to share an image with other accounts. Maybe your company has multiple accounts and you want to use a single corporate image across all accounts. Or maybe you have an image that includes a trial version of your company’s software and you want to share it with the world.

To share an image with another account, you use the Edit-EC2ImageAttribute command. In the following example, I am granting permission to launch an instance of an image to users of the account 1234-1234-1234. Obviously your image ID and account ID will be different.
Edit-EC2ImageAttribute -ImageId 'ami-71ebba18' -Attribute 'launchPermission'
     -OperationType 'add' -UserId '123412341234'
To share an image with all accounts, you grant permission to the group “all.” For example:
Edit-EC2ImageAttribute -ImageId 'ami-71ebba18' -Attribute 'launchPermission'
      -OperationType "add" -UserGroup "all"
You can check which accounts and groups have access by using the Get-EC2ImageAttribute command.
Get-EC2ImageAttribute -ImageId 'ami-71ebba18' -Attribute 'launchPermission'
To revoke the launch permission from an account, use the remove operation type. For example:
Edit-EC2ImageAttribute -ImageId 'ami-71ebba18' -Attribute 'launchPermission'
     -OperationType 'remove' -UserId '123412341234'
You can revoke the launch permission from the group the same way. For example:
Edit-EC2ImageAttribute -ImageId 'ami-71ebba18' -Attribute 'launchPermission'
     -OperationType 'remove' -UserGroup 'all'
If you want to revoke the launch permission from all users and groups, you can use the Reset-EC2ImageAttribute command. For example:
Reset-EC2ImageAttribute  -ImageId 'ami-71ebba18' -Attribute 'launchPermission'
Finally, if you are sharing images between accounts, you can list the images owned by a specific account by supplying the account number to the Get-EC2Image command. For example:
Get-EC2Image -Owner 123412341234

As you can see, AMIs are a powerful tool. You can leverage the tens of thousands of existing images, create your own images, and even share your images with others.

Although it is easy to customize an Amazon AMI, it would be great if we could leverage the library of images we already have onsite. In Exercise 7.1, I will show you how to import an existing VM image from an onsite hypervisor like VMware or Hyper-V.

Exercise 7.1: Uploading a VM

Many of us already have a library of images that we have built for our VMware or Hyper-V environments. Luckily Amazon allows you to upload an existing image into EC2.

There are a few ways to do this. If you have a lot of instances to import, you should look at the Server Migration Service (SMS). SMS is easiest way to import VMs into AWS. However, this is a book on PowerShell, so let's import an image using PowerShell.

Before you export the image, you should prepare the VM by checking the following:
  1. 1.

    Check that remote desktop is enabled.

     
  2. 2.

    Check that Windows Firewall allows public RDP traffic.

     
  3. 3.

    Ensure that DHCP is enabled.

     
  4. 4.

    Disable antivirus and intrusion detection systems.

     
  5. 5.

    Remove any virtualization tools such as the VMware Tools.

     
  6. 6.

    Disconnect any DVD (or other removable media) devices.

     
  7. 7.

    Stop the VM before importing it.

     

Now, it is time to export the image from your hypervisor. The import process supports VMDK, VHD, and OVF file formats. Let’s assume that our file is called GoldenImage.vmdk.

Next we upload the file to an S3 bucket. We are going to do this in two parts: first, upload the file to S3 and, second, import the image. Splitting it into two parts is really ingenious. It means that we could bulk import a bunch on VMs using an AWS Snowball. Snowball is a hardware appliance for importing (or exporting) large volumes of data through the mail.

We will cover S3 in detail in Chapter 11. For now, I will assume you have an S3 bucket called mybucket. You can upload the file like this:

Write-S3Object -File GoldenImage.vmdk -BucketName mybucket

Once the file is uploaded, we can import it. First we need to create a .Net object to describe our image, called an ImageDiskContainer. The DiskImageContainer tells the API the type of image and where to find the file in S3. For example:
    $container = New-Object Amazon.EC2.Model.ImageDiskContainer
    $container.Format="VMDK"
    $container.UserBucket = New-Object Amazon.EC2.Model.UserBucket
    $container.UserBucket.S3Bucket = "mybucket"
    $container.UserBucket.S3Key = "GoldenImage.vmdk"
Now we can import the image using the Import-EC2Image command like this:
Import-EC2Image -DiskContainer $container -ClientToken GoldenImage -Description "Golden Image" -Platform Windows -LicenseType AWS
As you might expect, the import operation takes a while. Import-EC2Image will return an ImportTaskId that you can use to check on the status. For example:
ec2-resume-import "c:awsMyImage.vhd" -t import-i-fh37272p
     -o %AWS_ACCESS_KEY% -w %AWS_SECRET_KEY%
Once the upload completes, Amazon will begin the conversion behind the scenes. There is no progress bar for this, but you can check on the conversion progress using ec2-describe-conversion-tasks command and passing your TaskId. For example:
ec2-describe-conversion-tasks import-i-fh37272p

Once the conversion completes, you will have an instance running in EC2 Classic. The import command does not clean up the temporary data stored in S3. You can delete it manually or use the ec2-delete-disk-image command .

Once your instance is imported, you can either use it as is or follow the instructions in this chapter to create derivative images. As you can see, the Import-EC2Image command will allow you to leverage your existing image library in the cloud and ensure that you have the same bits running on site and in the cloud.

Summary

In this chapter, we learned about Amazon Machine Images. We saw how to find and leverage the over 100,000 images already available. Then we discussed how to create our own custom images. We discussed how to prepare a Windows instance using SysPrep. Finally, we learned how to share our images with others and import images from our on-prem infrastructure.

Then, in the first exercise, we saw an alternative to rolling a custom image: scripted builds. In the second exercise, we saw how to import an existing image from VMware or Hyper-V. In the next chapter, we will talk about scalability and high availability.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset