SIRAJ CHAUDHARY: AWS Lambda

AWS Lambda is a serverless computing service that lets you run code without provisioning or managing servers. It automatically scales applications in response to traffic, making it ideal for certain workloads and architectures.

How AWS Lambda Works

Event-driven: Lambda runs your code in response to various events, such as an HTTP request (via API Gateway), changes in an S3 bucket, updates to a DynamoDB table, or periodic scheduled events (CloudWatch).

Functions: You write your code in the form of a "Lambda function," and Lambda executes this function in response to specific triggers.

Serverless: Lambda abstracts away server management. You don't need to worry about scaling, patching, or maintaining infrastructure.

Automatic scaling: Lambda automatically scales based on the number of incoming requests, adjusting the amount of compute power as needed.

Pay-per-use: You are charged only for the compute time consumed when your function is actively running (measured in milliseconds). There's no charge when your code is not running.

Key Features

Event Sources: Lambda integrates with many AWS services as event sources: S3, DynamoDB, SNS, SQS, CloudWatch Events, API Gateway, etc.

Supported Languages: AWS Lambda supports several programming languages, including Python, Node.js, Java, C#, Ruby, and Go. You can also create custom runtimes.

Stateless: Lambda functions are stateless by nature, which means each invocation is independent. If you need to persist data, you'd typically use external services like DynamoDB or S3.

Execution Context: AWS Lambda maintains a temporary execution environment (also called a "container") that can be reused for subsequent invocations, making subsequent calls faster (cold start vs. warm start).

Concurrency: Lambda can run multiple instances of your function in parallel (concurrent executions). AWS manages the scaling for you automatically.

Timeout: You can set a maximum execution time for your function (up to 15 minutes). If the function runs longer than the specified timeout, it will be terminated.

Layers: Lambda layers allow you to include external libraries, dependencies, or custom runtimes with your function without embedding them directly into the code package.

Pricing: You pay for the number of requests and the duration your code runs (measured in milliseconds). The first 1 million requests and 400,000 GB-seconds of compute time are free each month.

Common Use Cases

Data processing: Respond to changes in data (like S3 uploads or DynamoDB updates) in real-time.

Web applications: Use Lambda with API Gateway to create RESTful APIs.

Automated workflows: Trigger automated tasks based on events from other AWS services.

Cron jobs: Use Lambda with CloudWatch Events to run scheduled jobs without maintaining a server.

👉 Let's dive deeper into AWS Lambda and cover additional details such as architecture, deployment, performance optimization, security, and advanced use cases.

1. Architecture & Workflow

Lambda's architecture follows a simple flow:

Event Source: A service or system generates an event. For instance, an image is uploaded to S3, or an HTTP request is made via API Gateway.
Lambda Trigger: The event triggers the Lambda function to run. Lambda reads the event data and passes it to the function as input.
Execution Environment: Lambda runs your code in an isolated environment (container) that includes a specific runtime (Node.js, Python, Java, etc.). This environment is short-lived but can persist across invocations (warm starts).
Return or Response: The Lambda function processes the event and may return a response (e.g., HTTP response via API Gateway) or perform an action (e.g., write to a database)

2. Deployment and Management

Deploying and managing AWS Lambda functions can be done through multiple methods

AWS Management Console: Allows you to create, edit, and test functions directly from the web interface.

AWS CLI & SDKs: The AWS CLI (Command Line Interface) allows programmatic interaction with Lambda. You can deploy functions, update them, or invoke them from the command line.

Infrastructure as Code (IaC):

AWS CloudFormation: Helps automate the provisioning of Lambda functions alongside other AWS resources.

Serverless Framework: A popular open-source framework to define, deploy, and manage Lambda functions using configuration files (YAML).

AWS SAM (Serverless Application Model): SAM is an AWS-specific framework that simplifies the setup of Lambda-based applications using simplified CloudFormation syntax.

Terraform: A multi-cloud IaC tool that can also manage Lambda deployments.

3. Performance Optimization

1. Cold Start vs Warm Start

Cold Start: When a Lambda function is invoked for the first time (or after some idle time), AWS has to initialize a new execution environment (container). This involves downloading the code, initializing dependencies, and setting up the environment, which causes a slight delay (typically a few hundred milliseconds).

Warm Start: If a container from a previous invocation is available, Lambda reuses it, skipping the initialization process, leading to faster execution times.

Optimizations to Reduce Cold Start Time:

Use Smaller Packages: Keep your Lambda package size as small as possible to minimize the cold start initialization time.

Provisioned Concurrency: AWS allows you to keep a specific number of Lambda instances "warm" to avoid cold starts entirely (with an additional cost).

Avoid Heavy Initialization: Defer or reduce initialization (like opening database connections) until absolutely necessary, or use connection pooling with services like RDS Proxy.

2. Memory Tuning

Lambda allows you to configure memory allocation between 128 MB and 10 GB. AWS automatically allocates proportional CPU power based on memory.
Experiment with different memory settings to find the optimal balance between cost and performance (more memory often results in faster execution but costs more per invocation).

4. Security Considerations

IAM Roles and Policies: Lambda functions require execution roles with appropriate permissions to access other AWS services (e.g., S3 or DynamoDB). Always apply the principle of least privilege, ensuring the Lambda function only has permissions to do what it needs.

VPC Integration: Lambda can be configured to run inside a Virtual Private Cloud (VPC), giving it secure access to private resources (e.g., RDS databases that aren't publicly accessible). However, be aware that running Lambda inside a VPC may increase cold start times if it's not optimized.

Encryption: AWS automatically encrypts Lambda data at rest, and you can encrypt environment variables. For more sensitive data, you can use AWS KMS (Key Management Service) for additional encryption and control.

Environment Variables: Lambda supports environment variables, but ensure that sensitive data (e.g., API keys, credentials) is stored securely using encryption.

5. Monitoring and Debugging

Amazon CloudWatch: Lambda functions automatically log execution data to Amazon CloudWatch, where you can monitor and view:

Invocation counts
Duration (how long the function took to execute)
Errors (if any occurred during the function execution)
Memory usage
Logs of outputs, errors, and debugging information

AWS X-Ray: AWS X-Ray helps trace the flow of events and the performance of Lambda functions. It can help diagnose performance bottlenecks and trace requests as they pass through multiple AWS services.

6. Advanced Use Cases

Real-Time Data Processing: Use Lambda to process streaming data from services like Kinesis or DynamoDB Streams.
ETL (Extract, Transform, Load): Lambda is ideal for transforming and moving data between different AWS services, such as from S3 to Redshift, for data analysis pipelines.
Machine Learning Inference: You can use pre-trained machine learning models in Lambda for real-time inference (though for heavier workloads, using services like SageMaker might be more appropriate).
Chatbots and Voice Assistants: Lambda can be integrated with services like Amazon Lex and Alexa to build serverless voice or text-based bots.

7. Limitations

While AWS Lambda is powerful, it has some limitations:

Execution Time Limit: Lambda functions have a maximum execution time of 15 minutes.

Ephemeral Storage: Lambda provides temporary storage (/tmp directory) with up to 10 GB of space for each invocation, but this storage is wiped after the function completes.

File Size Limits: Lambda package deployment size is limited to 50 MB for zipped files (via direct upload) and 250 MB when deployed from S3.

No Long-Lived Connections: Since Lambda functions are stateless and short-lived, it may not be suitable for applications requiring persistent connections (e.g., WebSockets, long-running data streams).

8. Comparison with Other Compute Services

Lambda isn’t the only compute option in AWS. Here’s how it compares:

EC2 (Elastic Compute Cloud): EC2 provides virtual machines (VMs) where you manage the operating system and scaling. It is better suited for long-running processes or applications that need to control the underlying OS.

ECS / EKS (Elastic Container Service / Elastic Kubernetes Service): These are container-based compute services. Use these when you need finer control over container orchestration, scheduling, and networking.

AWS Fargate: Serverless containers (similar to Lambda but for containers). Fargate abstracts server management while allowing you to run Docker containers without worrying about the underlying infrastructure.

Conclusion

AWS Lambda is a versatile and powerful service ideal for event-driven, stateless, and short-lived workloads. Its automatic scaling, serverless nature, and pay-per-use pricing model make it a go-to choice for building modern, efficient cloud applications. However, for applications requiring long-running processes or persistent storage, alternatives like EC2, ECS, or Fargate may be more suitable.

Example-1: Create an AWS Lambda function in Java (using AWS SDKs) that will be triggered by an S3 bucket object change event (such as uploading or deleting an object)

Prerequisites: Install JDK, Maven, AWS CLI on your local system to create this project.

First, login to your AWS account with 'aws configure' command from AWS CLI

Step1: Create a S3 Bucket from AWS CLI

aws s3 mb s3://employer-s3-bucket --region ap-south-1

Step2: Set up a maven project for AWS Lambda

mvn archetype:generate -DgroupId=com.sirajchaudhary -DartifactId=employer-s3-lambda-handler-service -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false

Step3: Modify the pom.xml file to include the AWS Lambda and AWS SDK dependencies

<groupId>com.amazonaws</groupId>

<artifactId>aws-lambda-java-core</artifactId>

</dependency>

<groupId>software.amazon.awssdk</groupId>

</dependency>

<groupId>com.amazonaws</groupId>

<artifactId>aws-lambda-java-events</artifactId>

</dependency>

<groupId>com.amazonaws</groupId>

<artifactId>aws-lambda-java-log4j2</artifactId>

</dependency>

</dependencies>

<build>

<groupId>org.apache.maven.plugins</groupId>

<artifactId>maven-shade-plugin</artifactId>

<createDependencyReducedPom>false</createDependencyReducedPom>

</excludes>

</filter>

</filters>

</configuration>

<phase>package</phase>

<goals>

<goal>shade</goal>

</goals>

</execution>

</executions>

</plugin>

</plugins>

</build>

Step4: Implement the Lambda Function. Create a Java class, S3EventHandler, that will handle the S3 event

package com.sirajchaudhary;

import com.amazonaws.services.lambda.runtime.Context;

import com.amazonaws.services.lambda.runtime.RequestHandler;

import com.amazonaws.services.lambda.runtime.events.S3Event;

import software.amazon.awssdk.services.s3.S3Client;

import software.amazon.awssdk.services.s3.model.GetObjectRequest;

import java.io.InputStream;

public class S3EventHandler implements RequestHandler<S3Event, String> {

@Override

public String handleRequest(S3Event s3event, Context context) {

S3Client s3 = S3Client.builder().build();

s3event.getRecords().forEach(record -> {

String bucketName = record.getS3().getBucket().getName();

String objectKey = record.getS3().getObject().getKey();

context.getLogger().log("Received event for bucket: " + bucketName + ", object: " + objectKey);

// Example: Fetch the object content from S3

GetObjectRequest getObjectRequest = GetObjectRequest.builder()

.bucket(bucketName)

.key(objectKey)

.build();

try (InputStream objectData = s3.getObject(getObjectRequest)) {

// Process the object data (optional)

context.getLogger().log("Successfully processed object: " + objectKey);

} catch (Exception e) {

context.getLogger().log("Error processing object: " + e.getMessage());

}

});

return "Processed " + s3event.getRecords().size() + " records.";

}

This class will be invoked whenever an S3 event occurs. The handleRequest method receives an S3Event object, which contains details about the event such as the bucket name and object key.

Step5: Package your Lambda function as a .jar file

mvn clean package

Step6: Create the Lambda Function on AWS Management Console

Navigate to the AWS Lambda Console and create a new Lambda function.
Choose Author from scratch, set a function name, and choose the runtime as Java 17.
Upload the .jar file created in the previous step.
Set up the appropriate IAM role for your Lambda function with necessary permissions for S3.
In the Lambda Configuration section, under Triggers, add an S3 trigger:

Select the S3 bucket.
Set the event type to something like All object create events or any specific event you need (e.g., ObjectCreated).

Click Add trigger -> select source S3 and add your S3 details

Go to Code section -> Upload from -> .jar file -> upload your application jar

In Code section -> Runtime setting (Edit) -> here you have to add your handler method details e.g. com.sirajchaudhary.S3EventHandler::handleRequest

Step7: Test the Lambda Function

Upload a file to the S3 bucket you configured the trigger for.

👉 View logs in CloudWatch

Any logs or debug information will be available in the CloudWatch Logs associated with the Lambda function.

Go to the Lambda console -> click Monitor -> click View CloudWatch logs -> click Log stream ID -> see the logs and ensure the Lambda function was triggered correctly.

Summary

You created an AWS Lambda function in Java that listens to S3 object changes.
The Lambda function is triggered by S3 events and processes the object based on the event details.

👉 The Serverless Application Model (SAM)

AWS SAM is an open-source framework developed by Amazon Web Services (AWS) to simplify building, testing, and deploying serverless applications (e.g. AWS Lambda applications).

Here are some things you can do with AWS SAM

Build, test, debug, and deploy: Use the AWS SAM Command Line Interface (CLI) to build, transform, deploy, debug, package, initialize, and sync your application project directory.
Deploy gradually: AWS SAM can automatically create aliases for new versions of your Lambda function, and gradually shift traffic to the new version.
Roll back changes: If an update doesn't work, you can roll back the changes.
Verify configuration: AWS SAM can define pre- and post-traffic test functions to verify that your application is configured correctly.
Activate X-Ray tracing: You can activate X-Ray tracing for your Lambda functions.

AWS SAM is a specialized abstraction of CloudFormation that targets serverless applications. SAM simplifies the process of creating serverless applications with fewer lines of code compared to traditional CloudFormation templates.

AWS SAM uses a YAML file (or JSON) where you define your serverless application resources. These templates are an extension of AWS CloudFormation templates but are simplified to allow quick serverless development.

AWS SAM commands

sam build compiles and prepares your code for deployment.
sam package packages the application and uploads the artifacts to S3.
sam deploy simplifies deployment by managing CloudFormation stacks and resources.
sam deploy --guided is an interactive deployment process that asks questions to configure your deployment.
sam local invoke simulates Lambda function execution locally.
sam local start-api runs API Gateway locally for testing.
sam local start-lambda allows testing Lambda functions in a local Lambda environment.

AWS SAM supports integration with IDEs for debugging Lambda functions locally.

Typical workflow of AWS SAM

Write SAM Template: Define your serverless application in a template.yaml file.
Build: Run sam build to compile your functions and prepare the deployment package.
Test Locally: Use sam local invoke or sam local start-api to test your application.
Deploy: Use sam deploy --guided to deploy your application to AWS.

Advantages of AWS SAM

Simplifies Infrastructure as Code: Makes it easier to create serverless applications with CloudFormation.
Local Testing: Enables local testing of serverless apps, which can reduce development time.
Integration with AWS: Deep integration with AWS services such as Lambda, API Gateway, and CloudFormation.

Example-2: Create an AWS Lambda function in Java (using AWS SAM framework and AWS SDKs) which will be triggered by an API Gateway event (E.g. whenever the API Gateway URL will be hit the Lambda function will be triggered).

This Lambda function will be triggered by an API Gateway whenever API Gateway URL will be accessed. So in this Lambda function you can implement more logics like authentication, authorization for the incoming HTTP requests to API Gateway.

We use AWS SAM framework to create, build and deploy Lambda function project.

We use one AWS SAM framework's java template to create Lambda function project and use AWS SAM CLI to build and deploy the Lambda function into AWS. This template creates a boilerplate Lambda function which has a code to create an API Gateway once this get deployed into AWS. We will be able to access API Gateway with its given URL /hello. Whenever we will access the API Gateway URL the Lambda function will be triggered and the code inside Lambda function will be executed.

You can change the default project's code (Lambda function) provided by the template and can write your own Lambda functions and logics. To keep this simple we will not extend this project with more complex logics we will just generate a boilerplate code, build and deploy it into AWS using AWS SAM framework.

Prerequisites: Install JDK17, Maven, AWS CLI, AWS SAM on local system to create this project.

Note: Check which JDK versions get supported by your currently installed AWS SAM CLI using command "sam build --help" and use same JDK version on your local system while generating project template.

First, login to your AWS account with 'aws configure' command from AWS CLI

Step1: Create an AWS SAM Project using a java template with following command

sam init

When prompted:

Choose: 1 - AWS Quick Start Templates
Choose the project type: 1 - Hello World Example
Select the runtime: 8 - java17
..so on as shown in below snapshot. The project skeleton will be created as per these parameter.

This will create a basic SAM application with a Lambda function.

project skeleton is created with boilerplate code. Read "README.md" file which provide a bunch of command with definitions which you can run on this project.

In this boilerplate Lambda function, if you notice, there is a code which creates an API Gateway when we deploy Lambda function into AWS

Step2: Build & Deploy given boilerplate code as it is into AWS using AWS SAM CLI

mvn clean package

sam build

sam validate

sam deploy --guided

If it ask to create a S3 bucket at time of deployment than you can create it with follwoing command. This bucket will be used to save build artifacts.

aws s3 mb s3://<GIVEN-BUCKET-NAME>

The output provide many useful information like API Gateway URL which once we hit the Lambda function will be executed

https://at56rgx1gi.execute-api.ap-south-1.amazonaws.com/Prod/hello/

The project is deployed into AWS. A CloudFormation stack name sam-app is created. You can check created resources in API Gateway, Lambda console, CloudFormation, S3.

Step3: Test API

Go to API Gateway -> Select API (sam-app) -> Select Method (GET) -> Test

Step4: Clean up. Delete all resource (CloudFormation stack) associated with this project

sam delete --stack-name sam-app

Step5: You can customize given boilerplate Lambda function or create a new Labmda function as per your business needs and deploy it.

E.g. here is updated the Lambda function which is now a simple hello() function return a string.

You need to change unit tests in AppTest.java as per new logic of App.java.

You also have to update template.yaml as per your updated Lambda function

Now simply build maven project with command 'mvn clean package'

Than build and deploy SAM project with commands 'sam build' and 'sam deploy'

You can test deployed Lambda function in AWS Lambda console

Note: It did not create any API Gateway now as this time we didn't define it in Lambda function.

👉 Note: You can run and test this SAM framework project locally as well. So you don't need to deploy the application into AWS and than test. You need to have docker installed to fire following command and run the SAM application on your local system.

sam local invoke HelloWorldFunction

You can also pass parameters, if any, to your Lambada function using event.json file.

sam local invoke HelloWorldFunction --event events/event.json

Finally, delete the created stack completely

sam delete --stack-name sam-app

AWS Lambda

How AWS Lambda Works

Common Use Cases

1. Architecture & Workflow

2. Deployment and Management

1. Cold Start vs Warm Start

2. Memory Tuning

Views