© Brian Beach, Steven Armentrout, Rodney Bozo, Emmanuel Tsouris 2019
B. Beach et al.Pro PowerShell for Amazon Web Serviceshttps://doi.org/10.1007/978-1-4842-4850-8_18

18. Lambda with PowerShell

Brian Beach1 , Steven Armentrout2, Rodney Bozo3 and Emmanuel Tsouris4
(1)
Raleigh, NC, USA
(2)
Mountlake Terrace, WA, USA
(3)
Sterling, VA, USA
(4)
North Bend, WA, USA
 

With AWS Lambda you can deploy and execute code that can be triggered from a multitude of event sources without provisioning or the need to maintain any host servers. Lambda functions can run a number of different languages to include PowerShell Core. In this chapter, we’ll explain how to set up AWS Lambda using PowerShell and to execute PowerShell code as a Lambda function.

We’ll begin by going over prerequisites to set up your development environment and install required packages and modules. Then we’ll show you a couple ways to generate and deploy a PowerShell-Based Lambda function. We’ll walk through the execution of this Lambda function. We will evaluate the function execution and resultant output logs. Finally, we will set up an execution schedule for our new Lambda function.

There is one exercise at the end of this chapter. We’ll show you how to use AWS PowerShell on Lambda function to update an AWS Auto Scaling group ImageId that is triggered by an SNS notification.

To explore and learn more about the many features we’ll be talking about, open your browser and head over to the AWS Lambda console at https://console.aws.amazon.com/lambda .

PowerShell-Based Lambda Prerequisites

Lambda’s support for PowerShell is based on the cross-platform PowerShell Core. Because PowerShell Core is built on top of .NET Core, we will also need .NET Core installed. There are several new PowerShell publishing cmdlets to generate and deploy PowerShell-Based Lambda functions, and these require the .NET Core SDK.

We can download and install the latest .NET Core SDK from www.microsoft.com/net/download .

Once download and installation is complete, we need to install PowerShell Core. Downloads for various Linux and Windows installers can be found at https://github.com/PowerShell/PowerShell/releases .

Instructions for installing PowerShell Core on your specific development environment can be found at https://docs.microsoft.com/en-us/PowerShell/scripting/setup/installing-PowerShell?view=PowerShell-6 .

Once both the SDK and PowerShell Core are installed, we open a PowerShell Console session and verify the .NET Core SDK version installed meets the minimum requirements by executing the dotnet.exe with version parameter.
dotnet.exe --version

Note

Lambda support for PowerShell requires version 2.1 or greater of the .NET Core SDK.

We initialize the PowerShell Core executable from the PowerShell Console by executing pwsh.exe.
pwsh.exe
We can verify the current PowerShell version by reviewing the output of the $PSVersionTable variable.
write-output $PSVersionTable

Note

Lambda support for PowerShell requires PowerShell Core Edition 6.0 or greater. This version number can be seen as the PSVersion key value of the $PSVersionTable variable.

Next we install the AWS Lambda Core Module and import it into our current session. Confirm the request to install the NuGet package provider and trust the PSGallery repository modules if prompted. We then verify the AWS Lambda PowerShell Core module is found in the available module list using the Get-Module cmdlet.
Install-Module AWSLambdaPSCore -Scope CurrentUser -Force
Get-Module -ListAvailable -Name AWSLambdaPSCore
Finally, we complete the requirements by installing the AWSPowerShell.NetCore module from the PSGallery and verifying its version.
Install-Module AWSPowerShell.NetCore -MinimumVersion 3.3.270.0 -Scope CurrentUser –Force
Get-Module -ListAvailable -Name AWSPowerShell.NetCore

Note

Be sure to use version 3.3.270.0 or greater of AWSPowerShell.NetCore, which optimizes the cmdlet import process. If you use an older version, you will experience longer cold starts. To update your existing install to the latest version, run Update-Module AWSPowerShell.NetCore –Force.

The AWSPowerShell.NetCore module provides the SDK PowerShell cmdlets to interact with AWS infrastructure and services.

Authoring PowerShell-Based Lambda Functions

There are four new cmdlets available that we now have access to as a result of installing these prerequisites. Each of these cmdlets provides the tools necessary to quickly build, package, and deploy a PowerShell-Based Lambda function.

Creating a Script Template

First, we are going to list the common templates using the Get-AWSPowerShellLambdaTemplate cmdlet. A number of templates will be listed, and they can be distinguished by the type of trigger we expect to use for our Lambda function.
Get-AWSPowerShellLambdaTemplate
You should see a number of template listed as seen in Table 18-1.
Table 18-1

Lambda Templates

Template

Description

Basic

Bare bones script

CloudFormationCustomResource

PowerShell handler base for use with CloudFormation custom resource events

CodeCommitTrigger

Script to process AWS CodeCommit Triggers

DetectLabels

Use Amazon Rekognition service to tag image files in S3 with detected labels

KinesisStreamProcessor

Script to process a Kinesis Stream

S3Event

Script to process S3 events

S3EventToSNS

Script to process SNS Records triggered by S3 events

S3EventToSNSToSQS

Script to process SQS Messages, subscribed to an SNS Topic that is triggered by S3 events

S3EventToSQS

Script to process SQS Messages triggered by S3 events

SNSSubscription

Script to be subscribed to an SNS Topic

SNSToSQS

Script to be subscribed to an SQS Queue, that is, subscribed to an SNS Topic

SQSQueueProcessor

Script to be subscribed to an SQS Queue

We are now going to create a script named LatestAMIID using the basic template.
Set-Location $env:HOMEPATH
New-AWSPowerShellLambda -ScriptName LatestAMIID -Template Basic
Using Windows Explorer, when we look at our home directory we should now see a folder named LatestAMIID. The folder contains a corresponding named ps1 starter script and a Readme.txt file containing some additional details on the generated starter script. When we open the LatestAMIID.ps1 script template in the PowerShell ISE editor, we see content similar to Figure 18-1.
../images/319650_2_En_18_Chapter/319650_2_En_18_Fig1_HTML.jpg
Figure 18-1

Latest AMI ID template

Included in the script are a number of comments that will guide us in creating our PowerShell-Based Lambda function. In the next sections, we will investigate and understand the components of this template. We will also be adding some of our own code before saving it.

Understanding Modules

Near the top of the script template is the "#Requires" statement. This statement lists the modules that are needed by the PowerShell script and loads them when the Lambda function is executed. There can be many requires statements in a single script, and these modules must be packaged with your Lambda function for it to execute properly. These same statements are read by the AWS commands Publish-AWSPowerShellLambda and New-AWSPowerShellLambdaPackage that we will learn about later in this chapter. In our script templates, we see the following statement:
#Requires -Modules @{ModuleName='AWSPowerShell.NetCore';ModuleVersion='3.3.335.0'}

This statement indicates that the script requires the AWSPowerShell.NetCore module with a version of 3.3.335.0 or greater, we had installed earlier in this chapter. Note that 3.3.335.0 is the minimum version at the time of this books writing and any value greater than this in the script template is acceptable. For more information on the "#Requires" statement, see https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_requires?view=powershell-6 .

Understanding Input

At the top of the script template, we see how input is defined in a PowerShell-Based Lambda function. The $LambdaInput variable is a PSObject that contains all the data that is passed as input to the Lambda function when executed. This includes the data passed from triggering events. In Figure 18-2 we see that the SNS event trigger subject and message property values have already been assigned to the $subject and $message variables. We can easily adapt our code to perform logical operations from the $LambdaInput variable or one of its properties.

We may not know all the input properties and values from a triggering event so we will write them all as a JSON-formatted document upon execution of the Lambda function. To allow all input data to be written to CloudWatch Logs as output, we will uncomment or remove the # mark in front of the following statement in our script:
Write-Host (ConvertTo-Json -InputObject $LambdaInput -Compress -Depth 5)

Understanding Output and Logging

Lambda functions write their log data to CloudWatch Logs service. The write cmdlets that do not place data in the pipeline are written as logs. Cmdlets such as Write-Host and Write-Information as well as Write-Verbose and Write-Error if enabled are all written to CloudWatch Logs. Each CloudWatch Logs entry will also display its corresponding log level such as Information, Warning, or Verbose. See Figure 18-2 for example of CloudWatch Logs output.
../images/319650_2_En_18_Chapter/319650_2_En_18_Fig2_HTML.jpg
Figure 18-2

Lambda CloudWatch Logs

Write-Output places data into the pipeline so it does not get written to CloudWatch Logs. It shares a similar characteristic to the return statement in that it can be used in place of a return. Lambda function output will either be the last item added to the pipeline such as the last Write-Output statement or from a return statement if utilized. More details on Lambda function logging using PowerShell can be found at https://docs.aws.amazon.com/lambda/latest/dg/PowerShell-logging.html .

Understanding Errors

Lambda function executions end in either a succeeded or failed result. Failure to execute, timeouts, or script-level terminating errors are some of the ways that a Lambda function will result in a failed state. A successful execution of a Lambda function is all that’s needed for a passing result.

You can control the result of your Lambda function by using the throw or write-error statement in your script. The throw statement is a terminating error that will exit your script and Lambda function immediately. The write-error statement will continue executing your Lambda function script but will ultimately result in your function ending with a failed result. For more information on Lambda function errors in PowerShell, see https://docs.aws.amazon.com/lambda/latest/dg/PowerShell-exceptions.html .

The LambdaContext Variable

We also have the $LambdaContext variable listed at the top of our script templates. This variable provides runtime details about the overall execution of the Lambda function. This includes context object properties such as FunctionName, MemoryLimitInMB, and RemainingTime. Like the $LambdaInput object, the $LambdaContext is also an object that can be converted to a JSON document and written to CloudWatch Logs. For our testing purposes, we want to add the following line of code after the Write-Host for $LambdaInput statement in our script.
Write-Host (ConvertTo-Json -InputObject $LambdaContext -Compress -Depth 5)

Additional data on the Lambda Context Object for PowerShell can be found at https://docs.aws.amazon.com/lambda/latest/dg/PowerShell-context-object.html .

The script should now look similar to Figure 18-3.
../images/319650_2_En_18_Chapter/319650_2_En_18_Fig3_HTML.jpg
Figure 18-3

Log Only Template

Creating a PowerShell Lambda Package

Now we have a basic template script that can be used in a Lambda function, but does not perform any real actions other than importing modules and writing logs. Add the following lines of code to our LatestAMIID.ps1 script:
$VerbosePreference = "continue"
If ([string]::IsNullOrEmpty($LambdaInput.AliasName))
{
    [string]$AliasName = "Windows_Server-2016-English-Full-Base"
    Write-Warning "AliasName value missing. Using default Alias Name"
}
Else
{
    [string]$AliasName = $LambdaInput.AliasName
}
Write-information "Getting latest regional ImageId for public Amazon Machine Image with alias:$($AliasName)"
$ImageId=(Get-SSMParameter -Name /aws/service/ami-windows-latest/$($AliasName)).Value
Write-Verbose "Latest ImageId for Alias:$($AliasName) is:$($ImageId)"
return $ImageId
In this example script, we are going to return the latest Public ImagId for a matching alias name. You may recognize this code from Chapter 15 in the section titled “Finding the Latest Windows AMI.” Note that we are using a few different logging types for demonstration. Your final script should look similar to the following:
# PowerShell script file to be executed as a AWS Lambda function .
#
# When executing in Lambda the following variables will be predefined.
#  $LambdaInput - A PSObject that contains the Lambda function input data.
#  $LambdaContext - An Amazon.Lambda.Core.ILambdaContext object that contains information about the currently running Lambda environment.
#
# The last item in the PowerShell pipeline will be returned as the result of the Lambda function.
#
# To include PowerShell modules with your Lambda function, like the AWSPowerShell.NetCore module, add a "#Requires" statement
# indicating the module and version.
#Requires –Modules @{ModuleName='AWSPowerShell.NetCore';ModuleVersion='3.3.335.0'}
# Uncomment to send the input event to CloudWatch Logs
Write-Host (ConvertTo-Json -InputObject $LambdaInput -Compress -Depth 5)
Write-Host (ConvertTo-Json -InputObject $LambdaContext -Compress -Depth 5)
$VerbosePreference = "continue"
If ([string]::IsNullOrEmpty($LambdaInput.AliasName))
{
    [string]$AliasName = "Windows_Server-2016-English-Full-Base"
    Write-Warning "AliasName value missing. Using default Alias Name"
}
Else
{
    [string]$AliasName = $LambdaInput.AliasName
}
Write-information "Getting latest regional ImageId for public Amazon Machine Image with alias:$($AliasName)"
$ImageId=(Get-SSMParameter -Name /aws/service/ami-windows-latest/$($AliasName)).Value
Write-Verbose "Latest ImageId for Alias:$($AliasName) is:$($ImageId)"
return $ImageId

Now that we have a better understanding of PowerShell-Based Lambda functions script components and a completed script, save the changes and close it.

Before we can deploy our function, we first need to create a deployment package. There are a couple cmdlets to create a Lambda package. The first cmdlet Publish-AWSPowerShellLambda will create a temporary package and then publish it to Lambda automatically. The second cmdlet New-AWSPowerShellLambdaPackage will only create a named package we can deploy later. New-AWSPowerShellLambdaPackage is primarily used to generate packages as a part of an automated deployment package such as a CI/CD pipeline or as a part of a CloudFormation template. We are going to start with the New-AWSPowerShellLambdaPackage cmdlet so we can look at the generated package components and demonstrate a couple different deployment options.

To generate a Lambda package using our LatestAMIID.ps1 script, execute the following code in the PowerShell 6 Core Console we opened earlier in this chapter.
$LambdaPackage = New-AWSPowerShellLambdaPackage -ScriptPath .LatestAMIIDLatestAMIID.ps1 -OutputPackage .LatestAMIIDPackage.zip
Upon completion of the package creation, we see console output detailing the steps to creating the package. Make note of the reference to the handler value in the output. We will be using this value to deploy this package in future steps. We can also see this value by looking at the LambdaHandler property of the $LambdaPackage variable output as seen in Figure 18-4.
../images/319650_2_En_18_Chapter/319650_2_En_18_Fig4_HTML.jpg
Figure 18-4

New Lambda Package Output

Open the output package LatestAMIIDPackage.zip and review its contents. Among the many dll assemblies needed to host PowerShell Core, notice that our LatestAMIID.ps1 file is contained in this package as well as a folder labelled “Modules”. The modules folder contains the AWSPowerShell.NetCore module that is listed in our script requires statement. When we use either the New-AWSPowerShellLambdaPackage or Publish-AWSPowerShellLambda cmdlets, the requires statement is read from the script and those modules are placed into the packaged modules directory.

Publishing a PowerShell-Based Lambda Function

We are now ready to publish our new package. We will be working through a couple PowerShell-Based methods to publish a PowerShell Lambda package. The first method will use the package we just created and publish using some of the legacy AWS PowerShell Lambda cmdlets. The second method will use the newer Publish-AWSPowerShellLambda cmdlet.

Note

The remainder of this chapter will require administrative-level privileges to make modification to your AWS account. If you have not done so already, load your credentials in the current PowerShell 6 Core Console and set your default region to us-east-1. Lambda is a regional service, and the following chapter activities will be performed in the us-east-1 region.

Lambda functions require some privileges to access AWS resources. Required permissions will vary greatly depending on the Lambda function activities. For the sake of completion, we included the code to create a least privilege IAM role that will be assigned to our new Lambda function. To better understand the IAM role creation process, refer back to Chapter 2. Create a new IAM role with the name of “Lambda_LatestAMIID_Role” by using the following code:
$assumeRolePolicy = @"
{
    "Version": "2012-10-17",
    "Statement": {
        "Effect": "Allow",
        "Principal": {
            "Service": "lambda.amazonaws.com"
        },
        "Action": "sts:AssumeRole"
    }
}
"@
$rolepolicy = @"
{
    "Version": "2012-10-17",
    "Statement": [{
        "Effect": "Allow",
        "Action": [
            "logs:CreateLogStream",
            "logs:CreateLogGroup",
            "logs:PutLogEvents",
            "ssm:GetParameter"
        ],
        "Resource": "*"
    }]
}
"@
Import-Module AWSPowerShell.NetCore
$role = New-IAMRole -RoleName "Lambda_LatestAMIID_Role" -AssumeRolePolicyDocument $assumeRolePolicy
$newpolicy = New-IAMPolicy -PolicyName Lambda_LatestAMIID_Policy -PolicyDocument $rolepolicy
Register-IAMRolePolicy -rolename $role.RoleName -PolicyArn $newpolicy.arn
We now have an IAM role that only has write access to CloudWatch Logs and read access to AWS Systems Manager Parameter Store. To deploy the Lambda package we previously generated, we first assign a function name to the LMFunctionName variable that we will reuse.
$LMFunctionName = 'LatestAMIID'

Note

Due to the large number of parameters, many of the remaining examples in this chapter will pass parameters as a hashtable. This is known as PowerShell Splatting. More information on Splatting can be found at https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_splatting .

We will now use the Publish-LMFunction cmdlet and pass it a few parameters to create our new Lambda function using the following code:
$PublishLMParams = @{
"ZipFilename" = ".LatestAMIIDPackage.zip"
"Handler" = $LambdaPackage.LambdaHandler
"MemorySize" = 256
"Timeout" = 30
"Runtime" = "dotnetcore2.1"
"Role" = $role.arn
}
$PublishLM = Publish-LMFunction -FunctionName $LMFunctionName @PublishLMParams

Let’s spend a minute talking about some of the Publish-LMFunction parameters before continuing. The ZipFilename parameter identifies the location of our Lambda package zip. Handler defines what script Lambda calls as the entry point in our package to begin execution. If you recall, this value was defined when we created the Lambda package earlier in this chapter. The Role parameter gets assigned the ARN value of the IAM role we just created.

Timeout is the maximum amount of time the Lambda function can execute before force stopping with a failure result. Runtime is the environment our function will execute under. For PowerShell code we use the dotnetcore2.1 value.

MemorySize determines how performant our function will be as this value determines both available memory and the CPU power available but increases the cost of an execution. CPU power will effectively double every 128MB of memory allocated and can have considerable impact on how quickly your function executes. Too little memory can result in unexpected failures or hitting the function timeout, while too much results in costly unutilized capacity.

PowerShell modules and script executions require a bit more resources than the defaults allow. We increase the timeout from the default of 3 seconds to 30 seconds and increased the MemorySize from the default of 128 to 256.

For more details on configuration settings go to https://docs.aws.amazon.com/lambda/latest/dg/resource-model.html .

We now have successfully deployed a PowerShell-Based Lambda function. When we open the Lambda service in the AWS Management Console and select the LatestAMIID function, we see configuration data similar to Figure 18-5.
../images/319650_2_En_18_Chapter/319650_2_En_18_Fig5_HTML.jpg
Figure 18-5

Lambda Function Console View

While we are reviewing the console, let’s run a manual invocation. In the upper right corner of the LatestAMIID Lambda console window, select the Test button. When prompted by the Configure Test Event window, input “test1” into the event name box and click the Create button to close the window. The Lambda test console requires a Test Event configured before we can execute even if our function does not require any input.

In the upper right corner of the LatestAMIID Lambda function console, select the Test button again and the Lambda function will execute automatically. After a short time, you should see a green “Execution result: succeeded” message at the top. Clicking the arrow under this message expands the details similar to Figure 18-6.
../images/319650_2_En_18_Chapter/319650_2_En_18_Fig6_HTML.jpg
Figure 18-6

Lambda Execution Console View

Now that we know the function works, let’s run some PowerShell describe cmdlets to review the function configuration from the cmd line. Execute the following cmdlet:
Get-LMFunctionConfiguration -FunctionName $LMFunctionName

We see that the parameter values we used to publish the function match the console and Get-LMFunctionConfiguration output.

We are now going to remove this Lambda function and demonstrate deployment using the new Publish-AWSPowerShellLambda cmdlet. To remove the LatestAMIID Lambda function, execute:
Remove-LMFunction -FunctionName $LMFunctionName -Force

We executed with the force switch to automatically accept confirmation prompts. If we check the console or try the Get-LMFunctionConfiguration query again, we find the function no longer exists.

To deploy our script with Publish-AWSPowerShellLambda, we do not need the package we previously generated. This cmdlet automatically generates a package during the publication process for us. To redeploy our Lambda function, we execute the following example:
Publish-AWSPowerShellLambda -ScriptPath .LatestAMIIDLatestAMIID.ps1 -Name $LMFunctionName -IAMRoleArn $role.arn
Now let’s once again look at the deployed functions configuration using the following code:
Get-LMFunctionConfiguration -FunctionName $LMFunctionName

We can now see that a number of the properties we had to define earlier are automatically configured for us. The Handler, Runtime, Timeout, and MemorySize properties are all set automatically with default values that would allow executing a basic PowerShell script. Another nice feature of the Publish-AWSPowerShellLambda cmdlet is it can be run repeatedly to make a code change or update modules. It will automatically update the target Lambda function with whatever components that need changed.

Note

Publish-AWSPowerShellLambda cmdlet has a number of additional parameters to publish and update PowerShell-Based Lambda functions. For a full list, review the cmdlet help by executing “get-help Publish-AWSPowerShellLambda -full”.

Before we get into executing our Lambda function using PowerShell cmdlets, we will set the Timeout and MemorySize parameters to the values we originally set using the following command:
Update-LMFunctionConfiguration -FunctionName $LMFunctionName -MemorySize 256 -Timeout 30

Invoking Lambda Functions

Now that we have seen a couple ways to create and publish a PowerShell-Based Lambda function, let’s see how we can manually execute it and review the output.

Normally Lambda functions are invoked as a result of an event trigger such as an SNS notification, CloudWatch event, or write to an S3 bucket, to name a few. We can also invoke a Lambda function manually. This can be useful for debugging, testing or even creating our own code to initiate a Lambda function when appropriate.

The cmdlet we will be using to execute our newly published function is Invoke-LMFunction. First, let’s execute without any input payload by executing:
$ExecuteLMFunction = Invoke-LMFunction -FunctionName  $LMFunctionName  -LogType Tail
The LogType parameter with “Tail” value tells our script to wait for the Lambda function to complete and retrieves the last 4KB of log data written during the execution. To see these logs, we need to review the LogResult property of our execution. The LogResult data is Base64Encoding so we need to wrap it in a decoder by executing the following:
[System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($ExecuteLMFunction.LogResult))

We then see several lines of text where some are labelled as Information and others as Warning or Verbose. Remember that our code uses several different approaches to write log, and we can see here how Lambda automatically tags the log entry with the log level it was written under. Make note of the log entry labelled with Warning.

Now we are going to run our Lambda function again, but this time we are going to give it an input payload and capture its output data.

The Payload parameter allows us to pass JSON-formatted input to our Lambda function. In our example we will define an AMI AliasName with value as “Windows_Server-2012-R2_RTM-English-64Bit-Base” by executing the following parameter JSON coded for PowerShell:
$LMPayload = @"
{"AliasName": "Windows_Server-2012-R2_RTM-English-64Bit-Base"}
"@
We also add the InvocationType parameter. InvocationType can be set to one of three options:
  • RequestResponse – Executes and collects the output data upon completion

  • Event – Performs a quick asynchronous execution with no output data

  • DryRun – Used to test execution without actually executing the function

We will be using RequestResponse value for InvocationType so we can review the return data. Define the parameters and execute the Lambda function with the following code:
$ExecuteLMParams = @{
"FunctionName" = $LMFunctionName
"LogType" = "Tail"
"InvocationType" = "RequestResponse"
"Payload" = $LMPayload}
$ExecuteLMFunction = Invoke-LMFunction @ExecuteLMParams
Once again we decode the logs.
[System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($ExecuteLMFunction.LogResult))

We no longer see the Warning log entry and now see that one of the Information entries that was previously empty now contains data. Remember the "Write-Host (ConvertTo-Json -InputObject $LambdaInput -Compress -Depth 5)" and "Write-Host (ConvertTo-Json -InputObject $LambdaContext -Compress -Depth 5)" that we uncommented and added earlier to our script? These have correlating log entries we now see. LambdaInput object contains the data we passed as input with the Payload parameter. We also have another log entry that contains JSON-formatted data that includes FunctionName, LogGroupName, and MemoryLimitInMB, among many other attributes of the Lambda execution. This correlates to writing the LambdaContext object to logs.

The Lambda execution output data is assigned to the Payload property. If we look at $ExecuteLMFunction.Payload, we see that Payload is defined as a System.IO.MemoryStream. We cannot simply read it as a string value, so to read the output data, we wrap it in a StreamReader. Executing the following code will show a single ImageId value was returned.
([System.IO.StreamReader]::new($ExecuteLMFunction.Payload)).ReadToEnd()

Lambda CloudWatch Logs

As we know, all log outputs for Lambda functions are recorded in CloudWatch Logs automatically. There will be times when we will need to check the history of a Lambda execution or need more granular data to debug our function. While this is not specific to Lambda, it can be difficult to find relevant examples for searching CloudWatch Logs using PowerShell. We will filter for and return the CloudWatch Logs for our recent Lambda execution as a demonstration.

First we need to identify the RequestID for the last Lambda function execution we invoked. This is actually a property of our last invocation. It can be found under the ResponseMetadata property using the following query:
$LMRequestId = $ExecuteLMFunction.ResponseMetadata.RequestId

CloudWatch Logs are separated as log groups. All AWS Lambda functions write to the /aws/lambda/{Function Name} log group. Log groups are further broken down as log streams. Log streams in Lambda log groups are a collection of executions and their events that occurred within a time range.

To get our execution event logs, we need to filter through all of CloudWatch Logs and get the CloudWatch Logs log stream name that contains our recent Lambda execution request ID. We can only search through 1MB of logs per query so we use a do while loop to keep searching through the logs. The loop runs until we find a matching log entry or we complete searching through all log streams.

The following code returns the log stream events matched by our request ID string:
do {
    $MatchingLogs = (Get-CWLFilteredLogEvent –LogGroupName /aws/lambda/$LMFunctionName @args -FilterPattern "'"$($LMRequestId)'"")
    if ($MatchingLogs.Events){
        $LogStreamName = $MatchingLogs.Events[0].LogStreamName
        $LogStream = (Get-CWLLogEvent -LogGroupName /aws/lambda/$LMFunctionName -LogStreamName $LogStreamName)
        write-output $LogStream.Events | Format-Table –Wrap
        break
    }
    $args = @{"NextToken"=$([string]$MatchingLogs.Nexttoken)}
} While($MatchingLogs.Nexttoken)

Notice how we formatted the request ID as the FilterPattern value. This is because CloudWatch Logs filtering requires string matching searches to be in double quotes. We used escape characters to ensure that the double quotes were retained in the query. When we find matching logs, we identify the log stream name from the first matching log found and write the entire log stream as output. The matching log stream output will be formatted as a table so we can wrap the text to our screen for review.

Using this query, we can see the complete log for our Lambda execution.

Lambda Event Trigger

Up to this point, we have created a Lambda PowerShell script and packaged and published it. We also manually executed our Lambda function and evaluated output and log data. For our function to be useful, we also have to define a trigger. There are many Amazon services and conditions that can be used as triggers for Lambda functions. The configurations of these different triggers could easily span several chapters of their own. In this final topic and the chapter exercise, we will demonstrate two of the possible options.

Lambda service does not support schedules but the CloudWatch service does. In the following steps, we will be creating a CloudWatch Rule that will target our Lambda function every 60 minutes. We will then set a policy on our function allowing the 60 minute CloudWatch rule to invoke it.

Retrieve the ARN of our Lambda function by using the Get-LMFunctionConfiguration we executed earlier and retrieving the FunctionArn property.
$LambdaArn = (Get-LMFunctionConfiguration -FunctionName $LMFunctionName -Region us-east-1).FunctionArn

Now we create a CloudWatch Event Rule. We will define the name as “60MinuteTimer”. The ScheduleExpression parameter takes a rate or cron expression. In our example, we define a rate expression of 60 minutes. For details on Schedule Expressions, go to https://docs.aws.amazon.com/lambda/latest/dg/tutorial-scheduled-events-schedule-expressions.html .

The State parameter sets the Event Rule to either enabled or disabled. We intend to use this rule so we will set it to enabled with the following code:
$CWRuleName = "60MinuteTimer"
$CWRuleParams = @{
 ScheduleExpression = "rate(60 minutes)"
State = "ENABLED"
}
Now that we have defined our parameters, run the Write-CWERule cmdlet and evaluate that the $NewCWERule variable contains an ARN value for the new rule.
$NewCWERule = Write-CWERule -name $CWRuleName @CWRuleParams
Write-Output $NewCWERule
Set our Lambda function ARN as the target for the CloudWatch rule using the following code:
$CWEventTarget = New-Object Amazon.CloudWatchEvents.Model.Target
$CWEventTarget.Arn = $LambdaArn
$CWEventTarget.Id = (Get-random)
Write-CWETarget -Rule $CWRuleName -Target $CWEventTarget
We can verify the configuration for our CloudWatch event rule using the following two commands:
Get-CWERuleDetail -Name 60MinuteTimer
Get-CWETargetsByRule -Rule 60MinuteTimer
Finally, we need to set a policy on our Lambda function allowing our new 60MinuteTimer to invoke it. The Action parameter indicates that our function can be invoked. The Principal parameter is the source service being given permissions to invoke it. The SourceArn limits the invoke permissions to the 60MinuteTimer CloudWatch Rule. The FunctionName parameter limits the permissions to invoke our Lambda function ARN. The StatementId is a unique value we provide for this policy. We will fill it with a randomly generated number for this example. We now execute:
$LMPermissionParams = @{
Action = 'lambda:InvokeFunction'
Principal = "events.amazonaws.com"
SourceArn = $NewCWERule
FunctionName = $LMFunctionName
StatementId = (Get-Random)
}
Add-LMPermission @LMPermissionParams

All done. Our PowerShell-Based Lambda function will execute every 60 minutes. You can easily verify this by reviewing the function CloudWatch Logs for hourly entries.

Exercise 18.1: Update Auto Scale Group With Latest ImageId

In Chapter 8 we learned how to create an Auto Scaling group and corresponding Launch Configuration. In Chapter 15 we learned how to get the latest ImageId using SSM Parameter Store. In this exercise we will combine what we learned from those two chapters along with what we learned in this chapter to create a new Lambda function. This new function will automatically update our Auto Scale group with the latest ImageId whenever a new version is announced through a public SNS topic subscription.

Create the Script

First let’s identify the new Lambda function name and generate a script using the SNSSubscription template.
$FunctionName = "AutoUpdateASG"
New-AWSPowerShellLambda -ScriptName $FunctionName -Template SNSSubscription
Open the .AutoUpdateASGAutoUpdateASG.ps1 script in an editor and uncomment the following line of code:
Write-Host (ConvertTo-Json -InputObject $LambdaInput -Compress -Depth 5)
Add the following lines of code and save it:
$VerbosePreference = "continue"
[string]$AliasName = "Windows_Server-2012-RTM-English-64Bit-Base"
[string]$ASGroupName = "MyAutoScalingGroup"
Write-information "Getting latest regional ImageId for public Amazon Machine Image with alias:$($AliasName)"
$ImageId = (Get-SSMParameter -Name /aws/service/ami-windows-latest/$($AliasName)).Value
Write-Verbose "Latest ImageId for Alias:$($AliasName) is:$($ImageId)"
$MyASGroup = Get-ASAutoScalingGroup -AutoScalingGroupName $ASGroupName
$MyASLaunchConfig = Get-ASLaunchConfiguration -LaunchConfigurationName $($MyASGroup.LaunchConfigurationName)
$MyASLCImageID = $MyASLaunchConfig.ImageId
Write-Verbose "ImageId for Auto Scaling Group:$($ASGroupName) is:$($MyASLCImageID)"
If ($MyASLCImageID -ne $ImageId)
{
    $MyNewASLaunchConfigName = "MyLaunchConfig_$(Get-Date -format yyyy.MM.dd.hh.mm.ss)"
    Write-information "New ImageId available. Updating Auto Scaling group $($ASGroupName) with new Launch Configuration $($MyNewASLaunchConfigName)"
    $ASLaunchConfigParam = @{
    LaunchConfigurationName = $MyNewASLaunchConfigName
    ImageId = $ImageId
    KeyName = $MyASLaunchConfig.KeyName
    SecurityGroup = $MyASLaunchConfig.SecurityGroups
    Userdata = $MyASLaunchConfig.UserData
    InstanceType = $MyASLaunchConfig.InstanceType
    }
    $NewASLaunchConfig = New-ASLaunchConfiguration @ASLaunchConfigParam
    $UpdateASGroup = Update-ASAutoScalingGroup -AutoScalingGroupName $ASGroupName -LaunchConfigurationName $MyNewASLaunchConfigName
}

You can see in the script that we find the latest public ImageId from Parameter Store and compare it to the value set in the Auto Scale groups Launch Configuration we created in Chapter 8. The two ImageId values are compared, and if they do not match, a new Launch Configuration is created. The new Launch Configuration is then assigned to the Auto Scale group.

Create the IAM Role

Before we deploy this function, we need to create an IAM role with the required permissions. This new IAM role is an extension of the IAM role we created earlier in Chapter 18. The new role includes full control permissions for the AWS Auto Scaling service. Execute the following code to create our new IAM role:
$assumeRolePolicy = @"
{
    "Version": "2012-10-17",
    "Statement": {
        "Effect": "Allow",
        "Principal": {
            "Service": "lambda.amazonaws.com"
        },
        "Action": "sts:AssumeRole"
    }
}
"@
$labrole = New-IAMRole -RoleName "$($FunctionName)_Role" -AssumeRolePolicyDocument $assumeRolePolicy
$labrolepolicy = @"
{
    "Version": "2012-10-17",
    "Statement": [{
            "Effect": "Allow",
            "Action": [
                    "logs:CreateLogStream",
                    "logs:CreateLogGroup",
                    "logs:PutLogEvents",
                    "ssm:GetParameter"
"autoscaling:*"
            ],
            "Resource": "*"
    }]
}
"@
$newlabpolicy = New-IAMPolicy -PolicyName "$($FunctionName)_Policy" -PolicyDocument $labrolepolicy
Register-IAMRolePolicy -rolename $labrole.RoleName -PolicyArn $newlabpolicy.arn

Publish the Lambda Function

We now publish our Lambda function using the Publish-AWSPowerShellLambda cmdlet using the new IAM role we just created. We also set the memory and timeout values to ensure our function has enough time and resources to execute completely.
Publish-AWSPowerShellLambda -ScriptPath .AutoUpdateASGAutoUpdateASG.ps1 -Name $FunctionName -IAMRoleArn $labrole.Arn -Memory 512 -Timeout 10

Subscribe to the SNS Topic

Get the details of our new Lambda function and assign them to the $LMFunctionConfig variable.
$LMFunctionConfig = Get-LMFunctionConfiguration -FunctionName $FunctionName
Our Lambda function is ready so we subscribe its ARN to the public SNS topic used to announce new Amazon Machine Image Windows versions.
$SNSTopicARN = "arn:aws:sns:us-east-1:801119661308:ec2-windows-ami-update"
$Subscription = @{
  Protocol = 'lambda'
  Endpoint = $LMFunctionConfig.FunctionArn
  TopicArn = $SNSTopicARN
}
Connect-SNSNotification @Subscription

Note This is a public SNS topic used by AWS to announce when new versions of public Amazon Machine Images have been released. This occurs at least once a month, and more details can be found at https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/windows-ami-version-history.html .

Permit SNS Invocation

Finally, we give our Lambda function the permissions to be invoked by an SNS notification.
$LMPermission = @{
  FunctionName = $FunctionName
  Action = 'lambda:InvokeFunction'
  Principal = 'sns.amazonaws.com'
  StatementId = 1
}
Add-LMPermission @LMPermission

In this exercise we learned how to build and deploy a Lambda function that will update our Auto Scaling group with a new Amazon Machine Image. We also learned how to subscribe to and enable SNS as a trigger for our Lambda function.

Summary

In this chapter, we saw how to install the prerequisites for creating and deploying PowerShell-Based Lambda functions. We saw how to create a script, understand its components, and deploy it as a Lambda function. Using a few cmdlets, we saw how to execute our function and review the output as well as parse resulting CloudWatch Logs. Finally, we demonstrated how to create and assign a schedule trigger for our function. Lambda is a truly powerful tool to execute PowerShell code without owning or managing a single server.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset