Cold Starts in Serverless

Introduction

In January I gave a talk called From Dynamodb to RDS and Back: Database Considerations in a Serverless World at an Indy DevOps meetup. My talk broadly covered topics such as DynamoDB, RDS, cold starts, and databases connections in serverless. In this post I would like to take a deeper dive into cold starts. What are they, how do they affect performance, and how can they be minimized?

What is a Cold Start?

In a serverless world, the infrastructure that runs your code is not necessarily running all of the time. It is spun up and down based on demand. When a new instance is spun up to handle a request, it may take some extra time. This is extra time is called a cold start. Subsequent requests to the instance do not incur this initial time penalty. This is usually referred to the instance being warm. In AWS Lambda, cold start times can be broken down into two stages.

1. The time to spin up the Lambda to handle the request
2. The time to initialize your handler code

The picture above is of an AWS X-Ray trace of a Lambda that cold started. The 12 or seconds before Initializion was the time it took to spin up the Lambda to handle the request. The 1.4 seconds for Initialization was the time it took to initialize the code for the Lambda function. The combination of these two times is the total cold start time.

Spin Up Time

The total time to spin up a Lambda function to handle a request is usually outside of our control. Spin up times are usually no longer than a second or two. However if a Lambda is inside a VPC, this time will be longer. Lambdas inside a VPC use tohave spin up times as bad as 15 to 30 seconds. In November of last year AWS rolled out improved VPC networking for Lambda functions. This made cold start times better but still slower not being inside a VPC.

The moral of the story here is to limit the number of Lambdas inside a VPC. This is still necessary if your Lambda connects to AWS services that do no have IAM access control such as RDS.

Intialization

Intialization time is the time to initialize the code in your Lambda function to handle the request. This is any lines of code outside of your handler function. This is typically loading any external libraries and creating any objects that will stick around between requests.

const DynamoDB = require("aws-sdk/clients/dynamodb")

module.exports.handler = async event => {
  // do stuff with documentClient
}

For example, initialization time for the handler code above would be loading aws-sdk/clients/dynamodb and any of its references.

This time can be reduced by keeping your dependencies down. A great way to do this is to only have functions do one and only thing. Avoid branching paths or handling multiple types of events in a single function to keep initialization time down.

For javascript initialization time can also be reduced by minimizing code and using tree shaking to remove unused code.