In my post Creating an API with AWS, I demonstrated how to use a CloudFormation template to create a simple API with AWS. In this post, I’m going to walk through adding a DynamoDB database and POST method, and updating GET requests to pull data from the database.
Prerequisites
Before you start, you’ll need the following things:
- An AWS account
- An Administrator IAM User with which to use the AWS CLI
- AWS CLI set up on your local machine
- The CloudFormation template from my previous post, Creating an API with AWS
Note that AWS will charge you for creating and hosting this API if you don’t have Free Tier on your account.
Step 1: Add a DynamoDB Table to the CloudFormation Template
AWS DynamoDB is a fast, NoSQL database that is built around key-value pairs. For a simple web application, it is a good choice for data storage.
Taking the CloudFormation template (CFT) from my previous post, Creating an API with AWS, we can add a resource for a DynamoDB table which we’ll use to store the links under the Resources: line:
linkTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
-
AttributeName: "link_id"
AttributeType: "S"
-
AttributeName: "link"
AttributeType: "S"
KeySchema:
-
AttributeName: "link_id"
KeyType: "HASH"
-
AttributeName: "link"
KeyType: "RANGE"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
TableName: "linkTable"
This table will be called linkTable and have two attributes (fields): link_id, which will contain a unique id; and link, which will store the URL value. In the KeySchema section, we’ll define the link_id as the primary key and the link attribute as the sort key.
Step 2: Update the Lambda Execution Role in the CloudFormation Template
Next, we need to update the requestHandlerLambdaExecutionRole to enable the Lambda function to perform database operations. To do this, add the following policy underneath the existing policy named root:
-
PolicyName: "db_access"
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action:
- dynamodb:DeleteItem
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:Scan
- dynamodb:UpdateItem
Resource: !Sub "arn:aws:dynamodb:${AWS::Region}:${AWS::AccountId}:table/*"
This policy will allow the Lambda function to delete, get, add, and update database table items as well as perform full table scans.
Step 3: Update the Lambda Function to use the DynamoDB Database
Replace the requestHandler section of the CFT with the following Lambda function, which will check if a request is a GET or a POST, and store or retrieve data to/from the database as required:
requestHandler:
Type: "AWS::Lambda::Function"
Properties:
Code:
ZipFile: !Sub |
import json
import boto3
import uuid
# Establish credentials
session_var = boto3.session.Session()
credentials = session_var.get_credentials()
""" --- Parameters --- """
# DynamoDB tables, read lambda function environment variables or initialize with defaults if they do not exist.
DYNAMODB_LINK_TABLE = 'linkTable'
# Initialize DynamoDB Client
dynamodb = boto3.client('dynamodb')
def store_link(link):
link_id = str(uuid.uuid4())
dynamodb.put_item(
TableName=DYNAMODB_LINK_TABLE,
Item={
'link_id': {
'S': str(link_id)
},
'link': {
'S': str(link)
}
}
)
return link_id
def get_links():
links = []
result = dynamodb.scan(
TableName=DYNAMODB_LINK_TABLE,
Select="SPECIFIC_ATTRIBUTES",
ProjectionExpression="link_id,link"
)
if result['Count'] > 0:
for x in range(result['Count']):
link = {}
link['link_id'] = result['Items'][x]['link_id']['S']
link['link'] = result['Items'][x]['link']['S']
links.append(link)
return links
def error_response(error_message, aws_request_id):
return {
"statusCode": 500,
"isBase64Encoded": False,
"body": json.dumps({
"Error": error_message,
"Reference": aws_request_id,
}),
"headers": {
'Access-Control-Allow-Origin': '*',
},
}
def lambda_handler(event, context):
try:
print(event)
print(context)
return_body = ""
if event['httpMethod'] == "POST":
post_body = json.loads(event['body'])
link = post_body['link']
link_id = store_link(link)
return_body = json.dumps({"link_id": link_id})
elif event['httpMethod'] == "GET":
links = get_links()
return_body = json.dumps(links)
# Because this Lambda function is called by an API Gateway proxy integration
# the result object must use the following structure.
return {
"statusCode": 200,
"isBase64Encoded": False,
"body": return_body,
"headers": {
'Access-Control-Allow-Origin': '*',
},
}
except Exception as err:
print(str(err))
# If there is an error during processing, catch it and return
# from the Lambda function successfully. Specify a 500 HTTP status
# code and provide an error message in the body. This will provide a
# more meaningful error response to the end client.
error_response(str(err), context.aws_request_id)
Description: Lambda function for processing api endpoint requests
FunctionName: requestHandler
Handler: index.lambda_handler
Role:
Fn::GetAtt:
- "requestHandlerLambdaExecutionRole"
- "Arn"
Runtime: python3.8
DependsOn:
- requestHandlerLambdaExecutionRole
Step 4: Add a POST Method to the API in the CloudFormation Template
Add a POST method to the API so that new links can be added. To do this, add another AWS::APIGateway::Method resource under ProxyResourceGET called ProxyResourcePOST:
ProxyResourcePOST:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref link
HttpMethod: POST
AuthorizationType: NONE
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${requestHandler.Arn}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ApplicationAPI
Step 5: Launch the CloudFormation Template
Update the Description field to something that better describes what this CFT is building, e.g. “Creates a simple API endpoint that can be called to POST a link and store it in a database, or GET a list of links”. Save the CFT to a file called setupapidb.yml. The complete CFT should now look as below:
AWSTemplateFormatVersion: "2010-09-09"
Description: Creates a simple API endpoint that can be called to POST a link and store it in a database, or GET a list of links
Resources:
linkTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
-
AttributeName: "link_id"
AttributeType: "S"
-
AttributeName: "link"
AttributeType: "S"
KeySchema:
-
AttributeName: "link_id"
KeyType: "HASH"
-
AttributeName: "link"
KeyType: "RANGE"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
TableName: "linkTable"
requestHandlerLambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Principal:
Service:
- "lambda.amazonaws.com"
Action:
- "sts:AssumeRole"
Path: "/"
Policies:
-
PolicyName: "root"
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action:
- logs:CreateLogGroup
- logs:PutLogEvents
- logs:CreateLogStream
Resource: "*"
-
PolicyName: "db_access"
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action:
- dynamodb:DeleteItem
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:Scan
- dynamodb:UpdateItem
Resource: !Sub "arn:aws:dynamodb:${AWS::Region}:${AWS::AccountId}:table/*"
RoleName: requestHandlerLambdaExecutionRole
requestHandler:
Type: "AWS::Lambda::Function"
Properties:
Code:
ZipFile: !Sub |
import json
import boto3
import uuid
# Establish credentials
session_var = boto3.session.Session()
credentials = session_var.get_credentials()
""" --- Parameters --- """
# DynamoDB tables, read lambda function environment variables or initialize with defaults if they do not exist.
DYNAMODB_LINK_TABLE = 'linkTable'
# Initialize DynamoDB Client
dynamodb = boto3.client('dynamodb')
def store_link(link):
link_id = str(uuid.uuid4())
dynamodb.put_item(
TableName=DYNAMODB_LINK_TABLE,
Item={
'link_id': {
'S': str(link_id)
},
'link': {
'S': str(link)
}
}
)
return link_id
def get_links():
links = []
result = dynamodb.scan(
TableName=DYNAMODB_LINK_TABLE,
Select="SPECIFIC_ATTRIBUTES",
ProjectionExpression="link_id,link"
)
if result['Count'] > 0:
for x in range(result['Count']):
link = {}
link['link_id'] = result['Items'][x]['link_id']['S']
link['link'] = result['Items'][x]['link']['S']
links.append(link)
return links
def error_response(error_message, aws_request_id):
return {
"statusCode": 500,
"isBase64Encoded": False,
"body": json.dumps({
"Error": error_message,
"Reference": aws_request_id,
}),
"headers": {
'Access-Control-Allow-Origin': '*',
},
}
def lambda_handler(event, context):
try:
print(event)
print(context)
return_body = ""
if event['httpMethod'] == "POST":
post_body = json.loads(event['body'])
link = post_body['link']
link_id = store_link(link)
return_body = json.dumps({"link_id": link_id})
elif event['httpMethod'] == "GET":
links = get_links()
return_body = json.dumps(links)
# Because this Lambda function is called by an API Gateway proxy integration
# the result object must use the following structure.
return {
"statusCode": 200,
"isBase64Encoded": False,
"body": return_body,
"headers": {
'Access-Control-Allow-Origin': '*',
},
}
except Exception as err:
print(str(err))
# If there is an error during processing, catch it and return
# from the Lambda function successfully. Specify a 500 HTTP status
# code and provide an error message in the body. This will provide a
# more meaningful error response to the end client.
error_response(str(err), context.aws_request_id)
Description: Lambda function for processing api endpoint requests
FunctionName: requestHandler
Handler: index.lambda_handler
Role:
Fn::GetAtt:
- "requestHandlerLambdaExecutionRole"
- "Arn"
Runtime: python3.8
DependsOn:
- requestHandlerLambdaExecutionRole
ApplicationAPI:
Type: AWS::ApiGateway::RestApi
Properties:
Name: ApplicationAPI
Description: Simple API for an application
DependsOn:
- requestHandler
LambdaPermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !GetAtt
- requestHandler
- Arn
Action: 'lambda:InvokeFunction'
Principal: apigateway.amazonaws.com
SourceArn: !Join ["", [!Sub "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:", !Ref ApplicationAPI, "/*"]]
DependsOn:
- ApplicationAPI
link:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref ApplicationAPI
ParentId: !GetAtt [ApplicationAPI, RootResourceId]
PathPart: 'link'
DependsOn:
- ApplicationAPI
ProxyResourceGET:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref link
HttpMethod: GET
AuthorizationType: NONE
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${requestHandler.Arn}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ApplicationAPI
ProxyResourcePOST:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref link
HttpMethod: POST
AuthorizationType: NONE
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${requestHandler.Arn}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ApplicationAPI
OptionsMethod:
Type: AWS::ApiGateway::Method
Properties:
AuthorizationType: NONE
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref link
HttpMethod: OPTIONS
Integration:
IntegrationResponses:
-
StatusCode: 200
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
method.response.header.Access-Control-Allow-Methods: "'GET,POST,OPTIONS'"
method.response.header.Access-Control-Allow-Origin: "'*'"
ResponseTemplates:
application/json: ''
PassthroughBehavior: NEVER
RequestTemplates:
application/json: '{"statusCode": 200}'
Type: MOCK
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: true
method.response.header.Access-Control-Allow-Methods: true
method.response.header.Access-Control-Allow-Origin: true
DependsOn:
- ProxyResourceGET
Deployment:
DependsOn: "OptionsMethod"
Type: AWS::ApiGateway::Deployment
Properties:
RestApiId:
Ref: "ApplicationAPI"
Description: "Application API deployment"
StageName: "prod"
Outputs:
APIUrl:
Description: Application API Gateway link endpoint URL
Value: !Join
- ''
- - 'https://'
- !Ref ApplicationAPI
- '.execute-api.'
- !Ref AWS::Region
- '.amazonaws.com/prod/link'
Launch the CFT with the following command:
$ aws cloudformation create-stack --stack-name apidb --template-body file://setupapidb.yml --capabilities CAPABILITY_NAMED_IAM
Poll the progress of the CloudFormation stack with the following command:
$ aws cloudformation describe-stacks --stack-name apidb
Once complete, the value of “StackStatus” in the response should be “CREATE_COMPLETE” and you should have the API endpoint link in the “Outputs” section, under “OutputValue”. The link should be of the form: “https://xxxxxxxxxx.execute-api.your-region.amazonaws.com/prod/link”.
Step 6: Test the API
Test that everything is working by using curl. You may need to install curl first with sudo yum install curl or similar, depending on your operating system. Once installed, issue a curl POST request like the following. Be sure to replace the endpoint url with your own:
$ curl -v -X "POST" -H "Content-Type: application/json" -d "{\"link\": \"https://www.google.com\"}" https://xxxxxxxxxx.execute-api.your-region.amazonaws.com/prod/link
If the request is successful, you should see a response with a lot of connection information, and at the end of it, a line similar to this one:
{"link_id": "177173df-0e92-41a9-b2df-e8722443fd8a"}
That is the link_id of the link database entry just created.
Now try a GET request with the following curl request:
$ curl https://xxxxxxxxxx.execute-api.your-region.amazonaws.com/prod/link
If all is working, you should get a response like the below:
$ [{"link_id": "177173df-0e92-41a9-b2df-e8722443fd8a", "link": "https://www.google.com"}]
Try POSTing different links and then calling GET again. You should see the list of links getting larger.
Step 7: Clean Up
If you don’t want to keep your API (note that AWS will charge you for it if you don’t have Free Tier on your account), delete the CFT stack (all resources created) by running:
$ aws cloudformation delete-stack --stack-name apidb
You will then need to remove any logs that were generated by the Lambda function. To do this:
- Log into the AWS Console (your AWS account) using your Administrator IAM user, and navigate to CloudWatch.
- On the left-hand side, select Logs -> Log Groups.
- Select ‘aws/lambda/requestHandler’.
- Under Actions, select ‘Delete log groups’ and confirm.
Conclusion
You should now be able to create a simple API on AWS with a CloudFormation template, that stores and retrieves data to and from a DynamoDB database. If you’re interested in adding additional endpoints and databases, see my follow-on post, Creating an API with AWS: Part 3: Additional Endpoints.
Thanks for reading!
2 Responses
[…] Creating an API with AWS: Part 2: DynamoDB November 20, 2021 […]
[…] Creating an API with AWS: Part 2: DynamoDB November 20, 2021 […]