We will create a new script in our EffectiveDevOpsTemplates repository and call it vpc-cf-template.py.

We will start with our usual boilerplates:

"""Generating CloudFormation template.""" 
 
from troposphere import ( 
    GetAZs, 
    Output, 
    Parameter, 
    Ref, 
    Select, 
    Sub, 
    Tags, 
    Template, 
    GetAtt 
) 
 
from troposphere.ec2 import ( 
    VPC, 
    InternetGateway, 
    NetworkAcl, 
    NetworkAclEntry, 
    Route, 
    RouteTable, 
    Subnet, 
    SubnetNetworkAclAssociation, 
    SubnetRouteTableAssociation, 
    VPCGatewayAttachment, 
    EIP, 
    NatGateway, 
) 
 
t = Template() 
 
t.add_description("Effective DevOps in AWS: VPC, public and private subnets") 

This template will require providing a parameter for the CIDR. We will create our subnets on the private (non-publicly routable) IP address ranges 10.0.0.0/16 as specified in RFC 1918. Since we want to have the possibility to create multiple VPCs and have private subnets for each, we will limit our VPC to an a/16 network, which in most cases is enough, as it gives the ability to manage up to 65534 IPs:

t.add_parameter(Parameter( 
    "ClassB", 
    Type="Number", 
    Description="Class B of VPC (10.XXX.0.0/16)", 
    Default="0", 
    MinValue=0, 
    MaxValue=255, 
    ConstraintDescription="Must be in the range [0-255]", 
)) 

We will now create our resources. The first resource we will create is the VPC itself. The VPC resource has two optional attributes. The first one, EnableDnsSupport enables the ability to do DNS resolution through the AWS native DNS server at 169.254.169.253. EnableDnsSupport, the second attribute, makes it so that AWS will default to assigning DNS entries to instances launched in the VPC.

In addition to those attributes, we will also create a name tag such that when we list all the VPCs either with the command line or using the AWS console, it will be easier to differentiate the different VPCs created:

t.add_resource(VPC( 
    "VPC", 
    EnableDnsSupport="true", 
    EnableDnsHostnames="true", 
    CidrBlock=Sub('10.${ClassB}.0.0/16'), 
    Tags=Tags( 
        Name=Ref("AWS::StackName"), 
    ) 
)) 

The next resource we will create is an internet gateway. Internet gateways are used to proxy traffic between the instances in the VPC and the internet. Internet gateways are horizontally scaled, redundant, highly available, and completely managed by AWS making it a no-brainer to have if you want your instances to be able to reach outside of your VPC. Here too, we will tag them with the name of our stack to identify them easily in the InternetGateway list :

t.add_resource(InternetGateway( 
    "InternetGateway", 
    Tags=Tags( 
        Name=Ref("AWS::StackName"), 
    ) 
)) 

We have defined our VPC and internet gateway; now, we need to join the two together with a VPCGatewayAttachment resource as follows:

t.add_resource(VPCGatewayAttachment( 
    "VPNGatewayAttachment", 
    VpcId=Ref("VPC"), 
    InternetGatewayId=Ref("InternetGateway") 
)) 

We will now configure our network. Often when dealing with network configuration, this means instantiating and configuring a number of resources. Configuring our VPC will require creating the following:

1. Subnets, which are a range of IP addresses in the VPC which will be used to launch instances
2. Routing tables, which make it possible to configure where network traffic is directed in the subnets
3. SubnetRouteTableAssociation, which is, as its name suggests, the piece that connects subnets and routing tables together
4. Network access control lists (NACL), which let you control the traffic in and out of the subnets for an added layer of security
5. SubnetNetworkAclAssociation, which is similar to SubnetRouteTableAssociation but in that case, allows the association of NACL with subnets
6. NetworkAclEntry, which defines the rule inside the network ACL

The minimum required by AWS is to have one subnet per availability zone. In addition, we want to break out our hosts into publicly accessible or private examples. We need one subnet for each combination of AZs and accessibility (public or private). Since us-east-1 has four AZs this means creating two routing tables, two NACLs, eight subnets, 8SubnetRouteTableAssociation, 8SubnetNetworkAclAssociation resources, and four network ACL entries. If we were using CloudFormation, this would translate into a very lengthy repetitive and error-prone process to create those 32 resources. Fortunately, since we are creating a script to generate the CloudFormation template, we can take advantage of our situation and create several loops to minimize the work. We will first create three variables: an array called accessibility to break out private and public resources, names to help us differentiate the different AZs and finally, a simple counter:

accessibility = ["Private", "Public"] 
names = ["A", "B", "C", "D"] 
count = 0 

We will now create a loop through both accessibility values and create the routing tables and network ACL. We will proceed similarly as we did previously and add a Name tag for each of them to make their identification easier. For the purpose of our template, we will also programmatically generate their names and call them PrivateRouteTable, PublicRouteTable, PrivateNetworkAcl, PublicNetworkAcl, as follows:

for a in accessibility: 
    t.add_resource(RouteTable( 
        "{}RouteTable".format(a), 
        VpcId=Ref("VPC"), 
        Tags=Tags( 
            Name=Sub("${{AWS::StackName}} {}".format(a)), 
        ) 
    )) 
    t.add_resource(NetworkAcl( 
        "{}NetworkAcl".format(a), 
        VpcId=Ref("VPC"), 
        Tags=Tags( 
            Name=Sub("${{AWS::StackName}} {}".format(a)) 
        ) 
    )) 

We will now nest another loop inside the accessibility loop to list out each availability zone and create the subnets. Aside from the name that we will generate programmatically like we did for the previous resources, the creation of subnets requires specifying a number of parameters, including the AvailabilityZone, the CidrBlock, and specifying if the subnet is public or private. In order to provide the AvailabilityZone value, we will take advantage of the function GetAZs() which returns an array of all AZs available in a region (http://amzn.to/2nAtmsl), the function select which extracts a value from an array, and the counter variable previously initialized. This will allow us to distribute the different subnets we create evenly to each AZ. For the CIDR block, we will use the Class B parameter provided at the template creation time and also take advantage of the counter variable. We will multiply it by 16 such that we can create continuous blocks of 4094 hosts. Finally, to specify the value of the last parameter MapPublicIpOnLaunch, we will take advantage of the conditional statement functions (http://amzn.to/2oijikU). After the creation of the NetworkACL resources, add the following:

    for n in names: 
        t.add_resource(Subnet( 
            "{}Subnet{}".format(a, n), 
            VpcId=Ref("VPC"), 
            AvailabilityZone=Select(count % 4, GetAZs()), 
            CidrBlock=Sub("10.${{ClassB}}.{}.0/20".format(count * 16)), 
            MapPublicIpOnLaunch="true" if a == "Public" else "false", 
            Tags=Tags( 
                Name=Sub("${{AWS::StackName}} {} {}".format(a, n)), 
            ) 
        )) 

We can now increase the value of the counter and create the two remaining resources, SubnetRouteTableAssociation and SubnetNetworkAclAssociation as follows:

        count += 1 
        t.add_resource(SubnetRouteTableAssociation( 
            "{}Subnet{}RouteTableAssociation".format(a, n), 
            SubnetId=Ref("{}Subnet{}".format(a, n)), 
            RouteTableId=Ref("{}RouteTable".format(a)), 
        )) 
        t.add_resource(SubnetNetworkAclAssociation( 
            "{}Subnet{}NetworkAclAssociation".format(a, n), 
            SubnetId=Ref("{}Subnet{}".format(a, n)), 
            NetworkAclId=Ref("{}NetworkAcl".format(a)), 
        )) 

This completes the creation of our biggest loop and through it, the creation of most of the resources needed for our subnets. We just need to add our network ACL entries. For that, we will create another nested loop. This time, we will loop over the same accessibility and the network traffic origination. We will create a new array and call it directions as follows:

directions = ["Inbound", "Outbound"] 

From there, we will create our two for loops and add the NetworkAclEntries as follows:

for a in accessibility: 
    for d in directions: 
        t.add_resource(NetworkAclEntry( 
            "{}{}NetworkAclEntry".format(d, a), 
            NetworkAclId=Ref("{}NetworkAcl".format(a)), 
            RuleNumber="100", 
            Protocol="-1", 
            Egress="true" if d == "Outbound" else "false", 
            RuleAction="allow", 
            CidrBlock="0.0.0.0/0", 
        )) 

At that point, most of the complex resources are created. We need to add a few more things. We will first create a new route table to link our public route table to the internet gateway as follows:

t.add_resource(Route( 
    "RouteTablePublicInternetRoute", 
    GatewayId=Ref("InternetGateway"), 
    DestinationCidrBlock="0.0.0.0/0", 
    RouteTableId=Ref("PublicRouteTable"), 
)) 

We will now create our NAT gateway. Since it's preferable to always keep the same IPs for these types of services, we will first create an ElasticIP (http://amzn.to/2oi7v64) which will provide us with the ability to keep the same IP and reassign it to other resources if we need to rebuild our NAT gateway. For example:

t.add_resource(EIP( 
    "EIP", 
    Domain="VPC" 
)) 

We will now create the NAT gateway and provide it with the allocation ID from the EIP created just before. We will arbitrarily assign our gateway to our first public subnet:

t.add_resource(NatGateway( 
    "NatGateway", 
    AllocationId=GetAtt("EIP", "AllocationId"), 
    SubnetId=Ref("PublicSubnetA") 
)) 

With the creation of the NAT gateway, our private subnets now have a way to reach the internet. To complete this operation, we need to configure our route. We will do that by adding a new route table as we did a few lines before for the public route table as follows:

t.add_resource(Route( 
    "RouteNat", 
    RouteTableId=Ref("PrivateRouteTable"), 
    DestinationCidrBlock="0.0.0.0/0", 
    NatGatewayId=Ref("NatGateway") 
)) 

Our template is done. We will simply list our VPC ID in our CloudFormation template output and, as always, conclude the creation of the script by printing the JSON output of our template:

t.add_output(Output( 
    "VPCId", 
    Description="VPCId of the newly created VPC", 
    Value=Ref("VPC"), 
)) 
 
print(t.to_json())  

We can now commit our changes and generate the template and create a stack. We will pick the 10.10.0.0 network to create our VPC. As such, the command line to run is the following:

$ git add vpc-cf-template.py
$ git commit -m "Creating a new VPC with public and private zones"
$ git push
$ python vpc-cf-template.py > vpc-cf.template
$ aws cloudformation create-stack 
      --stack-name vpc-10 
      --capabilities CAPABILITY_IAM 
      --template-body file://vpc-cf.template 
      --parameters ParameterKey=ClassB,ParameterValue=10  

Our new VPC is now created. We will now redeploy our Hello World application to illustrate how to take advantage of this new architecture.