THE AWS CERTIFIED DEVELOPER – ASSOCIATE EXAM TOPICS COVERED IN THIS CHAPTER MAY INCLUDE, BUT ARE NOT LIMITED TO, THE FOLLOWING:
Now that you have an AWS account and you can make application programming interface (API) calls from your local machine, it is time to explore how to run code on the AWS Cloud. AWS provides a broad set of compute options through the following services:
In this chapter, you will explore Amazon EC2, which provides you with environments called instances. You will learn about the components of an Amazon EC2 instance and explore an example of customizing an instance to run an application. Then, to learn how to customize the network environment for your instances, you will explore the network controls of Amazon Virtual Private Cloud (Amazon VPC). Finally, you will review some of the concerns related to managing your compute and networking environments.
Amazon EC2 and Amazon VPC are foundational services, and many of the concepts introduced in this chapter are transferrable to working with other AWS services.
Amazon Elastic Compute Cloud (Amazon EC2) enables you to provision computing environments called instances. With Amazon EC2, you have the flexibility to choose the hardware resources you need. You are in control of the operating system and any other software that will run on the instance.
An Amazon EC2 instance runs on a host machine within a specific AWS Availability Zone. Typically, Amazon EC2 instances provide virtualized access to the underlying host machine resources. Using a combination of hardware and software components, instances present a virtualized interface to machine resources to the operating system. This virtualization enables multiple, different isolated guest environments to share the same underlying host machine. In addition to virtualized environments, some EC2 instance types offer bare-metal access. Bare-metal instances provide your applications with direct access to the processor and memory resources of the underlying server.
With Amazon EC2, you choose your hardware resources from a broad set of preconfigured options by selecting a specific instance type and instance size. For example, your instance has a number of virtual CPUs (vCPUs) and a specific amount of RAM. The instance type is rated for a certain level of network throughput. Some instance types also include other hardware resources such as high-performance local disks, graphics cards, or even field-programmable gate arrays (FPGAs). The details of how the instance accesses the host resources, such as the specific hypervisor in use, also depend on the instance type that you select.
Even though AWS presets the hardware allocation for an instance type, a wide variety of instance types and sizes are available so that you can select the right level of resources for your application. For example, a t2.nano instance type allocates a fraction of a virtual CPU and 0.5 GiB of RAM to your instance. On the other end of the size spectrum, an x1e.32xlarge instance type provides 128 virtual CPUs and 3,904 GiB of RAM.
Instance types are also grouped into instance families to help you choose the appropriate instance for your application. Instances within a given family share similar characteristics, such as the ratio of vCPU to RAM or access to different types of storage options.
For an overview of the different instance families and their use cases, see Table 2.1.
Table 2.1 Amazon EC2 Instance Families
Amazon EC2 Instance Family | For Applications That Require… |
General purpose | A balanced mix of CPU, RAM, and other resources |
Compute optimized | A high amount of CPU, such as high-performance web servers, scientific modeling, and video encoding |
Memory optimized | A large amount of RAM, such as in-memory databases and distributed web scale in-memory caches |
Storage optimized | A large amount of storage and input/output (I/O) throughput, such as data warehousing, analytics, and big data distributed computing |
Accelerated computing | Dedicated Graphics Processing Unit (GPU) or Field Programmable Gate Array (FPGA) resources, such as 3D rendering, deep learning, genomics research, and real-time video processing |
When you select an instance, choose a size that is appropriate for your current workload because Amazon EC2 instances are resizable. To change the hardware allocation, stop the instance, modify the instance type attribute, and then start the instance again.
Your instance requires storage volumes for both the root volume and any additional storage volumes that you want to configure. You can create persistent storage volumes with the Amazon Elastic Block Store (Amazon EBS) service to provide block storage devices for Amazon EC2 instances. Certain instance types enable you to mount volumes based on an instance store, which is temporary storage local to the host machine.
For an overview of the relationship between Amazon EBS volumes, instance store volumes, and the Amazon EC2 instance, see Figure 2.1.
For Amazon EC2 instances, Amazon EBS provides persistent block storage. Similar to a hard drive, block storage volumes provide read/write access at a block level and can be formatted with a file system. Also similar to a hard drive, you can attach each EBS volume to a single instance at a time. Amazon EBS is suitable for installing operating systems and applications and for data that you want to store persistently. You can also encrypt the volumes.
When you create an EBS volume, you provision a specific size for the storage volume. You choose from several types of volumes with different underlying storage technologies and performance options. You can increase the size of the volume later, even while it is being used by a running instance.
While an EBS volume is attached to a particular instance, only that instance can access the data on that volume. However, you can detach an EBS volume from one instance and then attach that volume to another instance in the same Availability Zone.
EBS volumes are decoupled from the underlying physical host running the instance. The decoupling of the storage volume from the host machine enables you to persist data even if your instance is no longer running on the physical host. Although the EC2 instance treats the EBS volume as a local disk, the underlying host machine reads and writes to the EBS volume over the network. To maintain peak performance for this connection, you can use EBS-optimized instance types. EBS-optimized instances reserve dedicated network bandwidth specifically for traffic to the EBS volume.
EBS volumes automatically replicate the data for a particular volume within the same Availability Zone as your Amazon EC2 instance. To increase durability of your data, you can use Amazon EBS to make point-in-time snapshots of an EBS volume. Data for Amazon EBS snapshots is automatically replicated across multiple Availability Zones within a region, and these snapshots can be used to create new volumes. If there’s an accidental delete or other application error, snapshots enable you to recover your data.
Certain Amazon EC2 instance types also allow you to mount instance store volumes—storage local to the physical host that runs your Amazon EC2 instance. An instance store volume is a good fit for high-performance storage of caches or temporary files and for use cases in which your application is already replicating the data to other locations.
This storage can have a high read/write performance because it is physically attached to the host machine that runs the instance. However, because this storage is local to the host machine, your data persists only while the instance is running on that host machine. The data persists if the instance reboots; however, AWS deletes the data on the instance store whenever you stop or terminate the instance.
When the server first boots, it requires an operating system (OS) and the configuration of the attached storage volumes. An Amazon Machine Image (AMI) provides the template for the OS and applications on the root volume of your instance. AMIs also provide a block device mapping that can specify additional volumes to mount when an instance launches, as shown in Figure 2.2.
AWS provides a variety of AMIs. Paid AMIs are available through the AWS Marketplace.
You can create your own AMIs from an Amazon EC2 instance that you have previously customized or by importing your own virtual machine (VM) images. Each AWS Region maintains its own listing of AMIs. Any AMIs that you create are available only within a specific region unless you copy them to other regions. You can share AMIs between AWS accounts. To control which AWS accounts can use your AMIs, define the launch permissions for your AMI.
Depending on the source of the AMI and the type of software license required, the cost of the software licensing may be included in the hourly rate of the instance (such as Windows Server). For instances from the AWS Marketplace, charges are incurred for software licensing in addition to the Amazon EC2 infrastructure.
Virtual network interfaces called elastic network interfaces provide networking for your Amazon EC2 instances. Elastic network interfaces are associated with a software-defined network provided by Amazon VPC. Each Amazon EC2 instance is assigned a primary network interface that is associated with a subnet within an Amazon VPC. By default, if you omit the network configuration, Amazon EC2 assigns the instance to one of the subnets within the default VPC. The instance receives both a private IP address to communicate with instances inside the Amazon VPC and a public IP address to communicate with the internet. A security group protects the traffic entering and exiting the network interface. Security groups act as a stateful firewall. To make network connections to your instance, you must set security group rules to allow the connection.
You can attach additional network interfaces to an EC2 instance. Each network interface has its own MAC addresses and IP address associations. Unlike the primary network interface, you can detach secondary network interfaces from one Amazon EC2 instance and then attach it to another instance.
The number of network interfaces that you can attach to an instance and the network throughput depends on the specific instance type and size that you select. The number of network interfaces that you attach does not affect the network throughput of the instance; the bandwidth available to the instance depends on the instance type and size, not the number of network interfaces.
By default, Linux Amazon EC2 instances provide remote access through SSH, and Windows Amazon EC2 instances provide remote access through the Remote Desktop Protocol (RDP). To connect to these services, you must have the appropriate inbound rules on the security group for the instance.
Depending on the operating system and AMI that you use to launch the instance, a default administrator is provided for your initial sign-in. To acquire the credentials needed to sign in as the default user, you must specify an Amazon EC2 key pair when you launch the instance. After you sign in, you can create additional users with the appropriate Linux or Windows tools.
The default user for Amazon Linux instances is ec2-user. For other Linux operating systems, this default user may vary depending on the AMI provider. For example, the default user for Ubuntu Linux is ubuntu.
For Windows instances, the default user is Administrator. This account may have a different name depending on the language of the server. For example, if the server is configured with French as the language, the administrator account is localized to Administrateur.
An Amazon EC2 key pair has a name, and it is composed of a public key and a private key. AWS retains the public key, and it is your responsibility to store the private key securely. If you specify an Amazon EC2 key pair when you launch the instance, it secures the sign-in credentials as part of the Amazon EC2 instance provisioning process. For a Linux instance, the public key from the key pair is added to the ~/.ssh/authorized_keys file for the default user. For a Windows instance, the password for the default administrator account is encrypted with the public key and can be decrypted with the private key.
When you create a new key pair, you can import a key pair that you generated locally, or you can have AWS generate a key pair for you. If you request that AWS generate the key pair, you can download the private key only at the time the key pair is generated. You are responsible for storing the private key file securely. You will not be able to download it again after it is created.
If you do not specify a key pair when you launch the instance, you are unable to sign in to that instance. Amazon EC2 key pairs are regional in scope, so you need key pairs in each region where you launch EC2 instances.
An Amazon EC2 instance has three primary states: running, stopped, and terminated. Additionally, there are intermediate states of pending, stopping, and shutting down. An Amazon EC2 instance accrues charges for the compute resources only when it is in the running state. However, EBS volumes persist data even when an instance is stopped, so the charges for persistent storage from any EBS volumes accrue independently from the state of the instance.
When you first launch an instance from an AMI, it goes into the pending state until it enters the running state on a host machine.
After an instance is running, for instances with EBS-backed storage, you can stop the instance. If you stop the instance, it enters the stopping state. Any data on instance store drives on that host are erased.
When an instance is stopped, you can modify attributes that cannot be changed, such as instance type, while the instance is running. You can also start stopped instances. When you start an instance, it enters the pending state until it is running again.
Typically, each time an instance is started, it is launched on a different physical host machine than before. If the underlying physical host is impaired and requires maintenance, stopping and then starting the Amazon EC2 instance moves the instance to a healthy host.
You can also terminate an instance. It first goes through shutting down; then eventually it is terminated. The default behavior is to delete the EBS volumes associated with the instance on termination.
To view the lifecycle of an Amazon EC2 instance, see Figure 2.3.
This section reviews how to connect to an EC2 instance and explores some features that are useful when you run applications or custom code on an instance. These features include ways of customizing the software on an instance, how your code can discover properties about the instance, and how to provide API credentials to your code running on an instance. An example that ties these features together is provided. Finally, the section describes how you can monitor the status of the instance.
With EC2 instances, you have full administrative control to install software packages on your instance and create additional user accounts as needed. By default, to connect to a Linux instance, you can directly use the private key from the Amazon EC2 key pair with an SSH client, as shown in Figure 2.4.
For a Windows instance, the password for the Administrator account is encrypted with the public key. You can decrypt the password by using the associated private key, as illustrated in Figure 2.5 and Figure 2.6.
After you have decrypted the password, you can use Microsoft Remote Desktop to connect to the instance, as shown in Figure 2.7.
You can connect to your instance and install any applications you want from an interactive session. However, one of the advantages of moving to the cloud is to automate previously manual steps. Instead of logging in to the instance, another way to customize the software on your instance is to provide user data as part of the request to launch the instance. For Linux instances, user data can be a shell script or a cloud-init directive. On Windows instances, depending on the version of Windows Server, either EC2Config or EC2Launch processes the user data. By default, commands supplied to user data execute only at first boot of the instance.
Here is an example of installing an Apache web server on an Amazon Linux 2 instance with a shell script that is provided as the user data:
#!/bin/bash
yum update -y
yum install httpd -y
systemctl start httpd
systemctl enable httpd
With the instance metadata service (IMDS), code running on an Amazon EC2 instance can discover properties about that instance. The instance metadata service exposes a special IP address, 169.254.169.254, which you can query using HTTP to perform lookups. You can query a broad range of metadata attributes, as shown in Figure 2.8 These attributes can include the instance ID and credentials derived from an Identity and Access Management (IAM) role.
With IMDS, it also possible to retrieve the user data that was used to bootstrap an instance, as shown in Figure 2.9.
Anyone who can access an instance can view its metadata and user data. Do not store sensitive data, such as passwords or access keys, as user data.
You can assign an IAM role to an Amazon EC2 instance. The AWS Software Development Kit (SDK) and AWS Command Line Interface (AWS CLI) can automatically discover these credentials through the Amazon EC2 metadata service. You can skip the task of explicitly configuring credentials files on your instances during bootstrapping.
When you assign an IAM role to an instance, it is assigned indirectly, through an instance profile, which is a container for an IAM role. You can associate a particular instance profile with many Amazon EC2 instances. However, a particular Amazon EC2 instance can be associated with one instance profile at a time, and an instance profile can be associated with only one IAM role. You can associate or disassociate Amazon EC2 instances with an instance profile at launch or even when the instances are running.
When an instance profile with an IAM role is associated with an instance, the Amazon EC2 service makes the necessary calls to the AWS Security Token Service (AWS STS) automatically to generate short-term credentials for that instance. These credentials are based on the IAM role associated with the instance profile. The credentials are exposed to the instance through the Amazon EC2 metadata service, as shown in Figure 2.10.
This example combines Amazon EC2 user data, the Amazon EC2 metadata service, and IAM roles to configure an Amazon EC2 instance. In the example, a web server displays a static webpage that shows custom information.
The following is a bootstrapping script that enables you to configure an Amazon EC2 instance running Amazon Linux 2. At first boot of the instance, the script generates a static page that displays the instance ID, instance type, Availability Zone, and public IP address of the instance at the time the script was executed.
The first line of the script declares the type of script. Then the script installs the Apache web server and configures it to run as a service. Because the script is running as the root user, there is no requirement to preface the commands with sudo.
Next, the script makes several calls to the Amazon EC2 metadata service and saves the results into environment variables to be used later in the script.
To generate an MP3 file that speaks out the instance ID, the script makes an API call to Amazon Polly. For this API call to succeed, you must assign the instance an IAM role that allows permissions to the Amazon Polly SynthesizeSpeech action. The managed AmazonPollyReadOnlyAccess policy can grant these permissions.
Next, the script generates a static HTML page. This page references the values that were previously stored as environment variables. After this script completes, your Amazon EC2 instance can respond to HTTP requests and show the customized page. To see this page, you must verify that the Amazon EC2 instance has port 80 open in its security group and is assigned a public IP address.
#!/bin/bash
# Install Apache Web Server
yum update -y
yum install httpd -y
systemctl start httpd
systemctl enable httpd
# Discover configuration using the EC2 metadata service
ID=$(curl 169.254.169.254/latest/meta-data/instance-id)
TYPE=$(curl 169.254.169.254/latest/meta-data/instance-type)
AZ=$(curl 169.254.169.254/latest/meta-data/placement/availability-zone)
IPV4=$(curl -f 169.254.169.254/latest/meta-data/public-ipv4)
# Set up the Web Site
cd /var/www/html
## Make AWS Cloud API calls to generate an audio file
VOICE=$(aws polly describe-voices --language-code en-US
--region us-west-2 --query Voices[0].Id --output text)
aws polly synthesize-speech --region us-west-2 --voice-id $VOICE
--text "Hello from EC2 instance $ID." --output-format mp3 instance.mp3
## Generate customized index.html for this instance
echo "<html> <body><H1>Welcome to your EC2 Instance</H1><p><p>" > ./index.html
echo "<audio controls>" >> ./index.html
echo '<source src="instance.mp3" type="audio/mp3">' >> ./index.html
echo 'Here is an <a href="instance.mp3"> audio greeting.</a> ' >> ./index.html
echo "</audio><p><p>" >> ./index.html
echo "There are many other instances, but" >> ./index.html
echo "<strong>$ID</strong> is yours.<p><p>" >> ./index.html
echo "This is a <strong>$TYPE</strong> instance" >> ./index.html
echo " in <strong>$AZ</strong>. <p><p>" >> ./index.html
if [ "$IPV4" ];
then
echo "The public IP is <strong>$IPV4</strong>.<p><p>" >> ./index.html
else
echo "This instance does <strong>NOT</strong> have" >> ./index.html
echo "a public IP address.<p><p>" >> ./index.html
fi
echo "--Audio provided by the $VOICE voice.<p><p>" >> ./index.html
echo "</body></html>" >> ./index.html
Now that you have an application running on your instance, you may be interested in understanding how that application performs, or if it is still running at all. Amazon EC2 performs automated status checks of the software and hardware of the underlying host machine every minute. These status checks are to verify that the instance can connect to the network and can run successfully on the host machine. If the status checks find no underlying issues, they return the status OK. If an issue is detected that prevents normal operation of the Amazon EC2 instance, the status checks return the status as impaired. The results of these status checks are available in Amazon CloudWatch.
For each of your instances, the Amazon EC2 service automatically collects metrics related to CPU utilization, disk reads and writes, and network utilization and makes them available in CloudWatch. You can supplement these built-in metrics with data from the guest operating system on your instance, such as memory utilization and logs from your application, by installing and configuring the CloudWatch agent on the instance.
Using CloudWatch, you can automate actions based on a metric through CloudWatch alarms. For example, you can configure an CloudWatch alarm that applies the recover instance action if status checks show that the host running the instance is impaired.
While the default Amazon VPC service provides a quick way to start with Amazon EC2 instances, it is important to understand how multiple Amazon EC2 instances communicate within an Amazon VPC network. The AWS Certified Developer – Associate exam may test your knowledge of Amazon VPC by asking troubleshooting questions related to the network. In this section, you’ll explore the Amazon VPC that enables you to create software-defined networks within an AWS Region.
Amazon Virtual Private Cloud (Amazon VPC) provides logically isolated networks within your AWS account. These networks are software defined and can span all of the Availability Zones within a specific AWS Region. For each VPC, you have full control over whether the Amazon VPC is connected to the internet, to a private on-premises network, or to other Amazon VPCs. Until you explicitly create these connections, instances in your VPC are able to communicate with other instances in the same VPC only.
You define an Amazon VPC with one or more blocks of addresses specified in the Classless Inter-Domain Routing (CIDR) notation. If, for example, you specified 10.0.0.0/16 as the block for a VPC, this means that the VPC includes IP addresses in the range from 10.0.0.0 through 10.0.255.255. For an example of an Amazon VPC spanning multiple Availability Zones in a region, see Figure 2.11.
By default, an Amazon VPC is an isolated network. Instances within an Amazon VPC cannot communicate with the internet or other networks until you explicitly create connections. Table 2.2 provides an overview of various types of connections that you can establish between an Amazon VPC and other networks.
Table 2.2 Amazon VPC Connection Types
Connection Type | Description |
Internet Gateway | A highly available connection that allows outbound and inbound requests to the internet from your Amazon VPC |
Egress Only Internet Gateway | A special type of internet gateway for IPv6 that allows outbound traffic and corresponding responses but blocks inbound connections |
Virtual Private Gateway | Allows you to establish a private connection to your corporate network by using a VPN connection or through Direct Connect (DX) |
Amazon VPC Endpoints | Allows traffic from your Amazon VPC to go to specific AWS services or third-party SaaS services without traversing an internet gateway |
Amazon VPC Peering | Privately routes traffic from one Amazon VPC to another Amazon VPC by establishing a peer relationship between this VPC and another VPC |
AWS Transit Gateway | Allows you to centrally manage connectivity between many VPCs and an on-premises environment using a single gateway |
For an example of an Amazon VPC with a connection to an internet gateway and a VPN connection to an on-premises network provided by a virtual private gateway, see Figure 2.12.
When working with Amazon VPC, all instances placed within a particular VPC are assigned one or more IP addresses. There are four different types of IP addresses available for use with Amazon VPC. Primarily, these IP addresses are based on IPv4; however, you can enable support for IPv6.
Private IP addresses are IPv4 addresses that are not reachable from the internet. These addresses are unique within a VPC and used for traffic that is to be routed internally within the VPC, for private communication with corporate networks, or for private communication with other VPCs.
When you create a VPC, you assign one or more blocks of addresses to the VPC, and typically these blocks will be within the range of IPv4 addresses reserved for private networks as specified in RFC1918. When an instance is launched, it is launched into a subnet within the VPC, and the instance is assigned a private IP address automatically from the block of addresses assigned to that particular subnet. When an instance is assigned a private IPv4 address, this association persists for the lifecycle of the instance—even when the instance is stopped.
Whether an EC2 instance is assigned public IP addresses automatically, in addition to the private IP address, depends on the following factors:
Unlike the private IP address, the public IP address is an IPv4 address that is reachable from the internet.
AWS manages the association between an instance and a public IPv4 address, and the association persists only while the instance is running. You cannot manually associate or disassociate public IP addresses from an instance.
An Elastic IP address is similar to a public IP address in that it is an IPv4 address that is reachable from the internet. However, unlike public IP addresses, you manage the association between instances and Elastic IP addresses. You control when these addresses are allocated, and you can associate, disassociate, or move these addresses between instances as needed.
You may also assign Elastic IP addresses to infrastructure such as NAT gateways. These addresses can come from a pool of IP addresses that AWS manages or from blocks of IPv4 addresses you have brought to your AWS account.
In addition to IPv4 addresses, you can associate an Amazon-provided block of IPv6 addresses to your VPC. When you enable IPv6 in your VPC, the network operates in dual-stack mode, meaning that IPv4 and IPv6 commutations are independent of each other. Your resources can communicate over IPv4, IPv6, or both.
Within an Amazon VPC, you define one or more subnets. A subnet is associated with a specific Availability Zone within the region containing the Amazon VPC. Each subnet has its own block of private IP addresses defined using CIDR notation. This block is a subset of the overall IP address range assigned to the Amazon VPC and does not overlap with any other subnet in the same Amazon VPC.
For example, a subnet may be assigned the CIDR block range 10.0.0.0/24, which would include addresses in the range 10.0.0.0–10.0.0.255. Out of the 256 possible addresses, Amazon VPC reserves the first four IP addresses and the last IP address in the range, leaving 251 IP addresses in the subnet.
When you launch an Amazon EC2 instance into a subnet, its primary network interface assigns a private IPv4 address automatically from the CIDR range assigned to the subnet.
Typically, you create at least two types of configurations for subnets in a VPC. The first is for subnets in which you place instances that you want to reach directly from the internet. This could be an instance running as a web server, for example. Subnets of this type are known as public subnets.
The second type of configuration is usually a subnet that backend instances use that must be accessible to your other instances but should not be directly accessible from the internet. Subnets of this type are known as private subnets. For example, if you had an instance that was dedicated to running a database, such as MySQL, you could place that instance in a private subnet. It would be accessible from the web server in the public subnet, but it would not accept traffic from the internet.
For an example of an Amazon VPC with a public and a private subnet, see Figure 2.13.
In addition to Amazon EC2 instances, many AWS managed services, such as Amazon Relational Database Service (Amazon RDS) or Amazon ElastiCache, also enable you to expose your resources in specific subnets and, in particular, into private subnets. You can create these resources and access them privately from instances within your Amazon VPC.
Network traffic exiting a subnet is controlled with routes that are defined in a route table. Routes define how the implicit router in the Amazon VPC routes IP traffic from a subnet to destinations outside that subnet. Each route table includes a rule called the local route. This rule or route is what allows traffic from instances in one subnet within the Amazon VPC to send traffic to instances in any other subnets within the same Amazon VPC. A route is composed of two parts: a destination and a target for the network traffic.
Unless explicitly associated with a specific route table, subnets associate with a default route table called the main route table. By default, the main route table includes only the local route. This means that subnets that are associated with the default route table have no connection to the internet. They can route traffic privately only within the Amazon VPC. However, you can modify this table or define additional route tables and rules as required.
For an example of the main route table for an Amazon VPC, see Table 2.3.
Table 2.3 Main Route Table Example
Destination | Target |
10.0.0.0/16 | local |
Route tables and the configured rules differentiate public subnets from private subnets. For example, you might create a public subnet by associating the subnet with a route table that includes a rule to route internet-bound traffic through an internet gateway. To represent any IP address on the internet in the rule, you can use the 0.0.0.0/0 CIDR block. Table 2.4 is an example of a route table that contains the defined rules.
Table 2.4 Public Route Table Example
Destination | Target |
10.0.0.0/16 | local |
0.0.0.0/0 | igw-example123 |
When you launch an Amazon EC2 instance into a public subnet, assign a public IP address to the instance. Even though the subnet has a route to an internet gateway, the instance is not able to communicate with the internet without a public IP address. Route table rules are evaluated in order of specificity.
To review a diagram of an Amazon VPC that has a public and a private subnet configured with the route table rules, see Figure 2.14.
Security groups act as a stateful firewall for your Amazon EC2 instances. When you define security group rules, you specify the source or destination of the network traffic in addition to the protocols and ports that you allow. If you change the security group rules, that change propagates to any instances associated with that security group.
By using inbound security group rules, you can control the source, protocols, and ports of allowed network traffic. For example, you could allow TCP connections that originate from the IPv4 address of your home network so that you can administer an Amazon EC2 instance using SSH.
Outbound rules enable you to control destination, protocols, and ports of allowed network traffic. Security groups include a default outbound rule that allows all outbound requests on all protocols and ports to all destinations. To control outbound requests more tightly, you can remove this default rule and add specific outbound rules in its place.
When you specify a source or destination for a security group rule, you can use IPv4 or IPv6 address ranges. Alternatively, you can use an identifier for a security group as a source or destination.
Assume that you have two EC2 instances—one instance is running a web server, and a second instance is running a database application. To allow network connections to these instances, you create two security groups: a security group for your web server instances called websg and a second security group for your database instances called databasesg.
For websg, you set inbound rules that allow web requests from anywhere. You also allow inbound SSH but only from a specific IP address that your administrator uses. You have not yet modified the default outbound rule for websg, so all outgoing connections are allowed.
For databasesg, you write an inbound rule that allows incoming traffic on TCP port 3306 originating from websg. Remove the default outbound rule and instead add rules to allow outbound connections to download software updates over HTTP and HTTPS. All other outbound connections from databasesg will be blocked.
To view the diagram of the security groups and rules for this scenario, see Figure 2.15. Also, see Table 2.5, Table 2.6, Table 2.7, and Table 2.8 for the corresponding inbound and outbound rules for these security groups.
Table 2.5 Inbound Rules for websg
Protocol | Port | Source | Comments |
TCP | 80 | 0.0.0.0/0 | Allow incoming HTTP requests from internet users |
TCP | 443 | 0.0.0.0/0 | Allow incoming HTTPS requests from internet users |
TCP | 22 | 10.10.0.6/32 | Allow incoming SSH only from the administrator’s computer |
Table 2.6 Outbound Rule for websg
Protocol | Port | Destination | Comments |
All | All | 0.0.0.0/0 | Allow all outbound IPv4 traffic |
Table 2.7 Inbound Rule for databasesg
Protocol | Port | Source | Comments |
TCP | 3306 | sg-123 | Allow inbound SQL queries from websg |
Table 2.8 Outbound Rules for databasesg
Protocol | Port | Destination | Comments |
TCP | 80 | 0.0.0.0/0 | Allow all outbound HTTP for updates |
TCP | 443 | 0.0.0.0/0 | Allow outbound HTTPS requests for updates |
If you specify an inbound rule, replies to that incoming connection are permitted. Similarly, if you specify an outbound rule, the replies to the outbound request are permitted. In the previous example, when a web server instance makes a connection to a database, only the outbound rule for the web server and the inbound rule for the database must allow the flow. The inbound rule for the web server and the outbound rule for the database will not be evaluated for this flow.
Security groups only support rules to allow traffic. Therefore, if you assign multiple security groups to your instance, the security group rules combine in the most permissive way; each group contributes to opening up more access to the instance.
If you fail to specify a security group when you launch the Amazon EC2 instance, the instance associates with the default security group for the Amazon VPC. If your Amazon EC2 instance has more than one network interface, you can manage the security groups for each network interface independently from the others.
In addition to routes, network access control lists (network ACLs) allow an administrator to control traffic that enters and leaves a subnet. A network ACL consists of inbound and outbound rules that you can associate with multiple subnets within a specific Amazon VPC. Network ACLs act as a stateless firewall for traffic to or from a specific subnet. Whereas security group rules provide only the capability to allow traffic, network ACL rules support the ability to allow specific types of traffic and to deny specific traffic.
However, unlike security groups, network ACLs are stateless and do not track connections and their replies. This means that to allow for a particular traffic flow, both inbound and outbound rules must allow it for that network ACL. For inbound rules, you can specify the protocol, port range, and source IP address range. For outbound rules, you specify the protocol, port range, and destination IP address range. For each rule, you also choose whether the rule allows or denies traffic. Rules in a network ACL are numbered and evaluated in order from the smallest to largest rule number.
If you do not specify a network ACL, the subnet is associated with the default network ACL for the Amazon VPC. This network ACL comes with rules that allow all inbound and outbound traffic. Table 2.9 shows an example of the inbound rules for a default network ACL. The final rule for the network ACL, rule 100, is a universal rule that explicitly denies traffic that does not match any other rule. Because there is a rule to allow all traffic and rules are evaluated in order, this universal rule has no effect. However, if you remove or modify rule 100, then the final rule would apply to any traffic that did not match any of the other rules.
Table 2.9 Default Network ACL Inbound Rules
Rule Number | Type | Protocol | Port Range | Source | Allow/Deny |
100 | All traffic | All | All | 0.0.0.0/0 | Allow |
* | All traffic | All | All | 0.0.0.0/0 | Deny |
Table 2.10 shows an example of the outbound rules for the default network ACL for an Amazon VPC. As before, the final rule is a universal rule that denies traffic unless it has been explicitly allowed by a preceding rule.
Table 2.10 Default Network ACL Outbound Rules
Rule Number | Type | Protocol | Port Range | Destination | Allow/Deny |
100 | All traffic | All | All | 0.0.0.0/0 | Allow |
* | All traffic | All | All | 0.0.0.0/0 | Deny |
Figure 2.16 shows an example of an Amazon VPC with security groups protecting Amazon EC2 instances and network ACLs protecting subnets.
Figure 2.17 shows the same Amazon VPC, represented in a different way to highlight the features that control network traffic within an Amazon VPC.
Table 2.11 summarizes key aspects of security groups and network ACLs.
Table 2.11 Security Groups and Network ACLs
Feature | Security Group | Network ACL |
Applies to | Amazon EC2 instance or elastic network interface. | Subnet |
Type of firewall | Stateful: Replies to an allowed traffic flow are automatically allowed. | Stateless: Must provide both inbound and outbound rules to allow a specific traffic flow. |
Rules | Only allow traffic. | Allow or deny traffic. |
Network address translation (NAT) allows for instances in a private subnet to make outbound requests to the internet without exposing those instances to inbound connections from internet users. To provide NAT for outbound requests from private subnets, you can use an Amazon EC2 instance configured to perform NAT or a NAT gateway. The instances in the private subnet maintain their own private IP addresses and effectively share the public IP address of the NAT when making internet requests.
For the NAT to perform its job, you must place the NAT instance or a NAT gateway in a correctly configured public subnet to forward traffic to the internet. Make sure that the public subnet has a route to an internet gateway, as previously shown in Table 2.4. To support outbound network requests, you can associate the private subnet with a route table, similar to the one shown in Table 2.12.
Table 2.12 Private Route Table Example
Destination | Target |
10.0.0.0/16 | local |
0.0.0.0/0 | nat-example456 |
For the Amazon VPC configuration example, Figure 2.18 shows the corresponding route tables for the public and private subnets using the NAT gateway.
Internet-bound requests route to the NAT gateway through the route table of the private subnet. The NAT, located in a public subnet, then makes a corresponding request out to the internet. This second outbound request appears to have originated from the public IP address of the NAT when the external website received it. When the website responds, NAT receives the reply and forwards it to the instance that initiated the original request. Figure 2.19 shows the network flow.
Using an Amazon EC2 instance for NAT instead of a NAT gateway requires you to disable the source/destination check setting for the NAT instance.
This setting protects Amazon EC2 instances by requiring that the instance be the source or destination of any network traffic it receives. In the case of the NAT instance, while the packets are routed to the instance via a route table, they are addressed to destinations on the internet rather than the NAT. Disabling this setting allows the network traffic to be delivered to the NAT instance.
The Dynamic Host Configuration Protocol (DHCP) provides a standard for passing configuration information to hosts on a TCP/IP network. The options field of a DHCP message contains the configuration parameters.
Some of those parameters are the address of domain name servers (DNS), the domain names of the instances, and the addresses of Network Time Protocol (NTP) servers. By default, the Amazon VPC uses the DNS that AWS provides. However, you can override these settings by specifying a custom set of DHCP options.
You can monitor the network flows within your Amazon VPC by enabling Amazon VPC Flow Logs. You can then publish these logs to Amazon CloudWatch Logs or store them as log files in Amazon Simple Storage Service (Amazon S3). Enable Amazon VPC Flow Logs on a particular Amazon VPC, on a subnet, or on a specific elastic network interface, such as for an Amazon EC2 instance.
For each network session, the Flow Logs capture metadata, such as the source, destination, protocol, port, packet count, byte count, and time interval. The log entry specifies whether the traffic was accepted or rejected. This information helps you debug the network configuration.
You now know that you can customize the software running on your Amazon EC2 instances. Additionally, with Amazon VPC, you can control the virtual network for those instances through security groups, network ACLs, subnets, and route tables. Because you can configure your operating environment in AWS, you share the responsibility for securing your applications running in the AWS Cloud with AWS.
AWS is responsible for the security of the cloud. This involves securing physical access to the underlying infrastructure, such as the AWS Regions and Availability Zones. This responsibility includes procedures for restricting access to the servers, physical networks, and decommissioning of hardware that is no longer useful. As part of securing the cloud infrastructure, AWS is also responsible for maintaining the underlying software for each service provided.
As the AWS customer, you are responsible for security on the cloud. This responsibility includes making secure choices when configuring your infrastructure and developing your applications. These responsibilities can include configuring the relevant encryption options and configuring your firewall rules. Even though this is your responsibility, you can simplify this task by taking advantage of AWS tools for encryption, defining firewall rules, and managing access and authorization to your AWS resources. For a summary of AWS and customer responsibilities, see Figure 2.20.
For example, with Amazon EC2, AWS is responsible for the software on the physical host machines up through maintaining the virtualization layer. However, beyond that, it is your responsibility to ensure that the guest operating system and everything running on it are secured. Your responsibilities include the following tasks:
AWS provides tools to help you manage these concerns. For example, use AWS Systems Manager to automate the patching of instances on your behalf. You can also use network controls, such as security groups, to restrict access to the instance. However, it is your responsibility to configure these features in a way that meets the security requirements for your specific application.
Even though services such as Amazon EC2 provide many low-level customizations, other AWS services provide a managed experience. When you use AWS managed services, you may find that you have less responsibility. For example, for Amazon RDS, AWS manages the software installed on the underlying database instance. You interact with the database using your SQL client rather than a shell session, and you do not install your own software on the database instance. In this case, parts of your operational burden for security are reduced—AWS manages the operating system and software on the database instance.
In addition to providing the low-level infrastructure pieces, such as Amazon EC2 instances and VPC networking components, the AWS Cloud provides higher-level services to help developers be more productive.
With AWS Cloud9, you can create developer environments that execute on either an Amazon EC2 instance or another server. AWS Cloud9 provides a web interface for editing code, debugging, and running commands. It supports more than 40 programming languages, and it can automatically configure the AWS SDK to use short-term managed credentials.
To access an AWS Cloud9 environment, sign in to the AWS Management Console. Within the AWS Cloud9 environment, the files you edit and run are on the remote instance or server. If you choose to run the AWS Cloud9 environment on an EC2 instance, you can customize the underlying EC2 instance as needed, even after the environment has been created. For example, you can edit the security groups of the instance or increase the size of the Amazon EBS volume attached to the instance.
This service simplifies the common task of developing code, running it on Amazon EC2, and collaborating with other developers. If you want to reduce cost, one advantage of AWS Cloud9 is that it can automatically stop the underlying instance a short time after you close your browser window, and it automatically starts the instance the next time you try to connect. You can further explore EC2 instances, VPCs, and AWS Cloud9 environments in the exercises that accompany this chapter.
Amazon Elastic Compute Cloud (Amazon EC2) instances are compute environments that provide you with full control over the operating system and software. The instance type and instance size determine the hardware available to an instance. This includes properties such as vCPU, RAM, access to local storage, and network bandwidth. Amazon Elastic Block Store (Amazon EBS) provides persistent storage for EC2 instances. An Amazon Machine Image (AMI) provides the template for the software on the instance. Additionally, user data allows you to run a script on the instance to automatically update the software on the instance. To make AWS API calls from code running on an EC2 instance, assign an AWS Identity and Access Management (IAM) role to the instance by way of an instance profile. Use Amazon CloudWatch to collect instance monitoring and utilization metrics.
Amazon Virtual Private Cloud (Amazon VPC) enables your EC2 instances to be placed into isolated networks where you have control over the connectivity to other networks, such as the internet, on-premises networks, or other VPCs. Within a VPC, the network is segmented into subnets. Instances within a subnet in a VPC are assigned private IPv4 addresses. They can be assigned public IPv4 addresses, Elastic IP addresses, or IPv6 addresses.
Routing between the instances in the VPC and other networks is controlled on a subnet level using routes and route tables. This configuration enables you to define some subnets as public and others as private. In addition to routing, network traffic can also be controlled by two sets of controls that act as firewalls in a VPC. Network access control lists (network ACLs) act as a stateless firewall on all traffic that leaves or enters a subnet. Security groups act as a stateful firewall that protects individual traffic flows at an instance level.
The responsibility for keeping your instances secure is shared between AWS and you, the customer. AWS is responsible for securing access to the infrastructure and providing you with controls that you can use to secure your instances. As an AWS customer, you are responsible for configuring your resources in a way that is secure and meets your application needs.
Know the basics of Amazon EC2, such as resource types, instance types, AMIs, and storage. Be familiar with launching and connecting to Amazon EC2 instances. Understand the resource types of Amazon EC2 instance types. Be familiar with the purpose of an AMI in relation to launching an instance. Understand the distinction between persistent and ephemeral storage related to a particular Amazon EC2 instance.
Know about user data, instance metadata, and credentials. Be familiar with using user data to customize the software by executing scripts on instances. Any scripts or code running on an instance can use the Amazon EC2 metadata service to discover the instance configuration. Use IAM roles to provide AWS Cloud API credentials automatically to code running on an Amazon EC2 instance.
Know how Amazon EC2 communicates with Amazon VPC. Understand the relationship between an EC2 instance and the Amazon VPC network. There may be questions that ask you to troubleshoot issues related to connecting to an Amazon EC2 instance. Be familiar with how Amazon VPC enables communication between Amazon EC2 instances within the same Amazon VPC and isolates those instances from other Amazon VPCs. Recognize how route tables, network access control lists, and security groups control network traffic.
Know about public and private subnets. Within an Amazon VPC, you must be able to distinguish between public and private subnets. Public subnets allow you to assign public IPv4 addresses to Amazon EC2 instances. By contrast, instances in a private subnet have only private IP addresses. The key distinction is that public subnets have a route table entry that forwards internet-bound traffic to an internet gateway. Private subnets do not have a direct route to the internet. Instead, these subnets have a route that forwards internet-bound traffic through a NAT gateway or NAT instance.
Know about security groups and network ACLs. Be familiar with security groups and network ACLs. Security groups are used with Amazon EC2 instances, acting as stateful firewalls. They provide only rules that allow traffic. In comparison, network ACLs allow traffic between subnets and are stateless. They can allow or deny specific types of traffic.
Know about responsibilities shared between you and AWS. Be familiar with the separation between AWS responsibility and your responsibility concerning Amazon EC2 instances. AWS is responsible for providing secure building blocks up until the hypervisor layer for the Amazon EC2 instance. This includes securing the physical facilities and machines and any hardware decommissioning. You are responsible for patching the guest operating system and applications. You are also responsible for configuring firewall rules, encryption, and access to the instance in a way that meets their requirements.
These exercises provide hands-on experience with the fundamentals of working with Amazon EC2 and Amazon VPC. You will create an isolated network in the AWS Cloud and then launch Amazon EC2 instances into that network. The exam has questions that test your knowledge of how to troubleshoot common network-connectivity issues relating to Amazon EC2 instances.
For the following exercises, verify that the region is US West (Oregon). The directions for these exercises assume that you have already completed Exercises 1.1, 1.2, 1.3, and 1.4 in Chapter 1, “Introduction to AWS Cloud API.”
You can complete these exercises within the AWS Free Tier, provided that you follow the steps to clean up resources promptly.
The results from these exercises are used in a later chapter, so follow all the activities and directions exactly.
In this exercise, you’ll generate and save an Amazon EC2 key pair. You are responsible for saving the private key and using it when you want to connect to your Amazon EC2 instances.
The key pair automatically downloads to your Downloads folder.
In this exercise, you’ll create an Amazon Virtual Private Cloud (Amazon VPC). Within that Amazon VPC, you will have a public subnet directly connected to the internet through an internet gateway. You will also have a private subnet that only has an indirect connection to the internet using network address translation (NAT).
VPC ID: VPC-06bb2198eaexample
Public subnet ID: subnet-0625e239a2example
Private subnet ID: subnet-0e78325d9eexample
In this exercise, you’ll create an IAM role for the web server. This role enables you to make AWS Cloud API calls from code running on the Amazon EC2 instance of the web server. You are not required to save IAM credentials in a file on the instance. To do this, create a new IAM role and call it the devassoc-webserver role. The role provides permissions needed for the API calls.
These permissions are required to complete future exercises.
In this exercise, you’ll launch an Amazon EC2 instance as a web server and connect to it.
#!/bin/bash
yum install httpd -y
systemctl start httpd
systemctl enable httpd
Paste this snippet from chapter-02/server-short.txt, located in the folder in which you downloaded the sample code for this guide.
The page fails to load because the security group rule allows HTTP access from only a particular IP address.
In this exercise, you’ll connect to the Amazon EC2 Instance using SSH.
curl 169.254.169.254/latest/meta-data/
curl 169.254.169.254/latest/meta-data/instance-id
aws translate translate-text --text "Hello world." --source-language-code en --target-language-code fr --region us-west-2
curl 169.254.169.254/latest/meta-data/iam/security-credentials/ devassoc-webserver
In this exercise, you’ll create a security group for the NAT instance. NAT allows Amazon EC2 instances in the private subnet to make web requests to the internet, to update software packages, and to make API calls.
In this exercise, you’ll launch an Amazon EC2 instance into the private subnet and then verify that the security group allows HTTP from anywhere. Because this instance is in the private subnet, it does not have a public IP address. Even though the instance can make outbound requests to the internet through the NAT instance, it is not reachable for inbound connections from the internet.
In this exercise, you will explore connectivity to the private instance.
curl private-ip-address
curl private-ip-address/instance.mp3 --output instance.mp3
sudo cp instance.mp3 /var/www/html/instance.mp3
http://public-ip-of-webserver/instance.mp3
Though the private web server is not reachable from the internet, you have confirmed that it is reachable to other instances within the same Amazon VPC. As part of the bootstrapping, the private instance made AWS API calls, which require the ability to make both web requests via the NAT gateway and credentials from an IAM role. You have confirmed that these requests succeeded by downloading the resulting MP3 file from private-instance and placing it on webserver.
In this exercise, you’ll launch an Amazon EC2 instance that you will create in the AWS Cloud9 service. You will connect to this Amazon EC2 instance from the AWS Cloud9 console. You will then use the AWS Cloud9 IDE to edit files, build software, and execute commands on the terminal from your web browser.
curl private-ip
curl private-ip-address/instance.mp3 --output instance.mp3
You now have a managed development environment in AWS that is connected to your isolated VPC.
In this exercise, you will clean up unused instances and keep this Amazon VPC for future use. This partial cleanup reduces costs while providing an environment to complete future exercises. After partial cleanup, you may generate charges related to the Elastic IP address that was allocated for devassoc-nat but is not in use while that instance is stopped.
Complete the following tasks as part of the cleanup:
If you plan to perform future exercises in this guide, this exercise is optional.
In this exercise, you will remove all of the EC2 and VPC resources that remain after Exercise 2.10.
If the Amazon VPC deletion fails, wait up to 30 minutes after deleting the Amazon EC2 instances and then try again.
When you launch an Amazon Elastic Compute Cloud (Amazon EC2) instance, which of the following is the most specific type of AWS entity in which you can place it?
You have saved SSH connection information for an Amazon Elastic Compute Cloud (Amazon EC2) instance that you launched in a public subnet. You previously stopped the instance the last time you used it. Now that you have started the instance, you are unable to connect to the instance using the saved information. Which of the following could be the cause?
You are working from a new location today. You are unable to initiate a Remote Desktop Protocol (RDP) to your Windows instance, which is located in a public subnet. What could be the cause?
You have a backend Amazon EC2 instance providing a web service to your web server instances. Your web servers are in a public subnet. You would like to block inbound requests from the internet to your backend instance but still allow the instance to make API requests over the public internet. What steps must you take? (Select TWO.)
You have launched an Amazon Elastic Compute Cloud (Amazon EC2) instance and loaded your application code on it. You have now discovered that the instance is missing applications on which your code depends. How can you resolve this issue?
How can code running on an Amazon Elastic Compute Cloud (Amazon EC2) instance automatically discover its public IP address?
How can you customize the software of your Amazon Elastic Compute Cloud (Amazon EC2) instance beyond what the Amazon Machine Image (AMI) provides?
You have a process running on an Amazon Elastic Compute Cloud (Amazon EC2) instance that exceeds the 2 GB of RAM allocated to the instance. This is causing the process to run slowly. How can you resolve the issue?
You have launched an Amazon Elastic Compute Cloud (Amazon EC2) Windows instance, and you would like to connect to it using the Remote Desktop Protocol. The instance is in a public subnet and has a public IP address. How do you find the password to the Administrator account?
What steps must you take to ensure that an Amazon EC2 instance can receive web requests from customers on the internet? (Select THREE.)
Which of the following are true about Amazon Machine Images (AMI)? (Select TWO.)
Which of the following are true about Amazon Elastic Compute Cloud (Amazon EC2) instance types? (Select TWO.)
Which of the following actions are valid based on the Amazon Elastic Compute Cloud (Amazon EC2) instance lifecycle? (Select TWO.)
You have a development Amazon Elastic Compute Cloud (Amazon EC2) instance where you have installed Apache Web Server and MySQL. How do you verify that the web server application can communicate with the database given that they are both running on the same instance?
What type of route must exist in the associated route table for a subnet to be a public subnet?
What type of route must exist in the associated route table for a subnet to be a private subnet that allows outbound internet access?
Which feature of Amazon Virtual Private Cloud (Amazon VPC) enables you to see which network requests are being accepted or rejected in your Amazon VPC?
Which AWS service enables you to track the CPU utilization of an Amazon Elastic Compute Cloud (Amazon EC2) instance?
What happens to the data stored on an Amazon Elastic Block Store (Amazon EBS) volume when you stop an Amazon Elastic Compute Cloud (Amazon EC2) instance?
Which programming language can you use to write the code that runs on an Amazon EC2 instance?
You have launched an Amazon EC2 instance in a public subnet. The instance has a public IP address, and you have confirmed that the Apache web server is running. However, your internet users are unable to make web requests to the instance. How can you resolve the issue? (Select TWO.)
Which of the following are the customer’s responsibility concerning Amazon EC2 instances? (Select TWO.)