In my post Creating an API with AWS: Part 2: DynamoDB, I updated the simple AWS API endpoint with a DynamoDB database table for data storage. In this post, I’ll add two additional endpoints, with two additional database tables. One of the tables will be a linking table between the two other tables. This allows data stored in one table to be linked to data stored in another. In this example, we’ll add a new endpoint called tag, which we can use to manage tags, and a third endpoint called taglink, which we can use to assign tags to links.
Prerequisites
Before you start, you’ll need the following things:
- An AWS account
- An Administrator IAM User with which to use the AWS CLI
- AWS CLI set up on your local machine
- The CloudFormation template from my previous post, Creating an API with AWS: Part 2: DynamoDB.
Note that AWS will charge you for creating and hosting this API if you don’t have Free Tier on your account.
Step 1: Add/Update DynamoDB tables in the CloudFormation Template
Taking the CloudFormation template (CFT) from my previous post, Creating an API with AWS: Part 2: DynamoDB, update the linkTable resource with a Global Secondary Index. This index will be used to find entries by link value. Add it under the TableName attribute.
GlobalSecondaryIndexes:
-
IndexName: "link_index"
KeySchema:
-
AttributeName: "link"
KeyType: "HASH"
-
AttributeName: "link_id"
KeyType: "RANGE"
Projection:
ProjectionType: "KEYS_ONLY"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
Next, add two new resources for a DynamoDB tables (tagTable and tagLinkTable) which will be used to store the tags and tag-link mappings. Add them under the linkTable resource:
tagTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
-
AttributeName: "tag_id"
AttributeType: "S"
-
AttributeName: "tag"
AttributeType: "S"
KeySchema:
-
AttributeName: "tag_id"
KeyType: "HASH"
-
AttributeName: "tag"
KeyType: "RANGE"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
TableName: "tagTable"
GlobalSecondaryIndexes:
-
IndexName: "tag_index"
KeySchema:
-
AttributeName: "tag"
KeyType: "HASH"
-
AttributeName: "tag_id"
KeyType: "RANGE"
Projection:
ProjectionType: "KEYS_ONLY"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
tagLinkTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
-
AttributeName: "tag_id"
AttributeType: "S"
-
AttributeName: "link_id"
AttributeType: "S"
KeySchema:
-
AttributeName: "tag_id"
KeyType: "HASH"
-
AttributeName: "link_id"
KeyType: "RANGE"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
TableName: "tagLinkTable"
Note the the tagTable resource also has a Global Secondary Index to enable filtering by tag name.
Step 2: Update the Lambda Role Permissions in the CloudFormation Template
Next, update the requestHandlerLambdaExecutionRole resource, adding a permission to run database queries to the list of existing database permissions:
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action:
- dynamodb:DeleteItem
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:Scan
- dynamodb:UpdateItem
- dynamodb:Query
This will enable the lambda function to run filtered queries on the database tables.
Step 3: Update the Lambda Function in the CloudFormation Template
Update the requestHandler resource with the new lambda code needed to handle the endpoint requests and database queries:
import json
import boto3
import uuid
# Establish credentials
session_var = boto3.session.Session()
credentials = session_var.get_credentials()
""" --- Parameters --- """
# DynamoDB tables, read lambda function environment variables or initialize with defaults if they not exist.
DYNAMODB_LINK_TABLE = 'linkTable'
DYNAMODB_TAG_TABLE = 'tagTable'
DYNAMODB_TAG_LINK_TABLE = 'tagLinkTable'
# Initialize DynamoDB Client
dynamodb = boto3.client('dynamodb')
def store_link(link):
link_id = str(uuid.uuid4())
dynamodb.put_item(
TableName=DYNAMODB_LINK_TABLE,
Item={
'link_id': {
'S': str(link_id)
},
'link': {
'S': str(link)
}
}
)
return link_id
def store_tag(tag):
tag_id = str(uuid.uuid4())
dynamodb.put_item(
TableName=DYNAMODB_TAG_TABLE,
Item={
'tag_id': {
'S': str(tag_id)
},
'tag': {
'S': str(tag)
}
}
)
return tag_id
def store_taglink(tag_id, link_id):
dynamodb.put_item(
TableName=DYNAMODB_TAG_LINK_TABLE,
Item={
'tag_id': {
'S': str(tag_id)
},
'link_id': {
'S': str(link_id)
}
}
)
def get_link_id(link):
result = dynamodb.query(
TableName=DYNAMODB_LINK_TABLE,
IndexName='link_index',
KeyConditionExpression='link = :link',
ExpressionAttributeValues={
':link': {
"S": link
}
}
)
if result['Count'] > 0:
link_id = result['Items'][0]['link_id']['S']
else:
link_id = store_link(link)
return link_id
def get_tag_id(tag):
result = dynamodb.query(
TableName=DYNAMODB_TAG_TABLE,
IndexName='tag_index',
KeyConditionExpression='tag = :tag',
ExpressionAttributeValues={
':tag': {
"S": tag
}
}
)
if result['Count'] > 0:
tag_id = result['Items'][0]['tag_id']['S']
else:
tag_id = store_tag(tag)
return tag_id
def get_tags(tag_id=None):
tags = []
if tag_id is None:
result = dynamodb.scan(
TableName=DYNAMODB_TAG_TABLE,
Select="SPECIFIC_ATTRIBUTES",
ProjectionExpression="tag_id,tag"
)
else:
# Run a query with a filter on tag_id on the tagTable
result = dynamodb.query(
TableName=DYNAMODB_TAG_TABLE,
KeyConditionExpression='tag_id = :tag_id',
ExpressionAttributeValues={
':tag_id': {
"S": tag_id
}
}
)
if result['Count'] > 0:
for x in range(result['Count']):
tag = {}
tag['tag_id'] = result['Items'][x]['tag_id']['S']
tag['tag'] = result['Items'][x]['tag']['S']
tags.append(tag)
return tags
def extract_link_result(result):
links = []
if result['Count'] > 0:
for x in range(result['Count']):
link = {}
link['link_id'] = result['Items'][x]['link_id']['S']
link['link'] = result['Items'][x]['link']['S']
links.append(link)
return links
def get_links(tag_id=None, link_id=None):
links = []
link_id_list = []
if tag_id is None and link_id is None:
result = dynamodb.scan(
TableName=DYNAMODB_LINK_TABLE,
Select="SPECIFIC_ATTRIBUTES",
ProjectionExpression="link_id,link"
)
links = extract_link_result(result)
elif link_id is not None:
# Run a query with a filter on link_id on the linkTable
result = dynamodb.query(
TableName=DYNAMODB_LINK_TABLE,
KeyConditionExpression='link_id = :link_id',
ExpressionAttributeValues={
':link_id': {
"S": link_id
}
}
)
links = extract_link_result(result)
else:
# Run a query with a filter on tag_id on the tagLinkTable
result = dynamodb.query(
TableName=DYNAMODB_TAG_LINK_TABLE,
KeyConditionExpression='tag_id = :tag_id',
ExpressionAttributeValues={
':tag_id': {
"S": tag_id
}
}
)
if result['Count'] > 0:
for x in range(result['Count']):
link_id_list.append(result['Items'][x]['link_id']['S'])
# From the retrieved link_ids, retrieve the links from the link table
if len(link_id_list) > 0:
for link_id in link_id_list:
result = dynamodb.query(
TableName=DYNAMODB_LINK_TABLE,
KeyConditionExpression='link_id = :link_id',
ExpressionAttributeValues={
':link_id': {
"S": link_id
}
}
)
if result['Count'] > 0:
link = {}
link['link_id'] = link_id
# There should only be one result per link_id
link['link'] = result['Items'][0]['link']['S']
links.append(link)
return links
def error_response(error_message, aws_request_id):
return {
"statusCode": 500,
"isBase64Encoded": False,
"body": json.dumps({
"Error": error_message,
"Reference": aws_request_id,
}),
"headers": {
'Access-Control-Allow-Origin': '*',
},
}
def bad_request_response(error_message, aws_request_id):
return {
"statusCode": 400,
"isBase64Encoded": False,
"body": json.dumps({
"Error": error_message,
"Reference": aws_request_id,
}),
"headers": {
'Access-Control-Allow-Origin': '*',
},
}
def lambda_handler(event, context):
try:
print(event)
print(context)
return_body = ""
if event['resource'] == '/link':
if event['httpMethod'] == "POST":
post_body = json.loads(event['body'])
link = post_body['link']
tag = post_body.get('tag')
tag_id = post_body.get('tag_id')
if tag_id is None:
tag_id = get_tag_id(tag)
else:
tags = get_tags(tag_id)
if not tags:
return bad_request_response('Unknown tag_id ' + tag_id, context.aws_request_id)
link_id = get_link_id(link)
store_taglink(tag_id, link_id)
return_body = json.dumps({"link_id": link_id})
elif event['httpMethod'] == "GET":
tag_id = None
link_id = None
query_params = event['queryStringParameters']
if query_params is not None:
tag_id = query_params.get('tag_id', None)
tag = query_params.get('tag', None)
link_id = query_params.get('link_id', None)
if tag_id is None and tag is not None:
tag_id = get_tag_id(tag)
links = get_links(tag_id, link_id)
return_body = json.dumps(links)
elif event['resource'] == '/tag':
if event['httpMethod'] == "POST":
post_body = json.loads(event['body'])
tag = post_body.get('tag')
tag_id = get_tag_id(tag)
return_body = json.dumps({"tag_id": tag_id})
elif event['httpMethod'] == "GET":
tag_id = None
query_params = event['queryStringParameters']
if query_params is not None:
tag_id = query_params.get('tag_id', None)
tag = query_params.get('tag', None)
if tag_id is None and tag is not None:
tag_id = get_tag_id(tag)
tags = get_tags(tag_id)
return_body = json.dumps(tags)
elif event['resource'] == '/taglink':
if event['httpMethod'] == "POST":
post_body = json.loads(event['body'])
tag_id = post_body.get('tag_id')
link_id = post_body.get('link_id')
tags = get_tags(tag_id)
links = get_links(tag_id=None, link_id=link_id)
if not tags:
return bad_request_response('Unknown tag_id ' + tag_id, context.aws_request_id)
if not links:
return bad_request_response('Unknown link_id ' + link_id, context.aws_request_id)
store_taglink(tag_id, link_id)
return_body = json.dumps({"tag_id": tag_id, "link_id": link_id})
# Because this Lambda function is called by an API Gateway proxy integration
# the result object must use the following structure.
return {
"statusCode": 200,
"isBase64Encoded": False,
"body": return_body,
"headers": {
'Access-Control-Allow-Origin': '*',
},
}
except Exception as err:
print(str(err))
# If there is an error during processing, catch it and return
# from the Lambda function successfully. Specify a 500 HTTP status
# code and provide an error message in the body. This will provide a
# more meaningful error response to the end client.
return error_response(str(err), context.aws_request_id)
Step 4: Add Two New API Resources in the CloudFormation Template
Add two API Gateway resources, one for the tag endpoint, and one for the taglink endpoint.
tag:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref ApplicationAPI
ParentId: !GetAtt [ApplicationAPI, RootResourceId]
PathPart: 'tag'
DependsOn:
- ApplicationAPI
taglink:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref ApplicationAPI
ParentId: !GetAtt [ApplicationAPI, RootResourceId]
PathPart: 'taglink'
DependsOn:
- ApplicationAPI
Step 5: Add Methods for the New Endpoints in the CloudFormation Template
Add methods for the tag and taglink endpoints. Update the Deployment resource to depend on the OptionsMethodTagLink resource, which should be the last endpoint method to be created, with the rest in a chain.
ProxyResourceTagGET:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref tag
HttpMethod: GET
AuthorizationType: NONE
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${requestHandler.Arn}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceLinkPOST
ProxyResourceTagPOST:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref tag
HttpMethod: POST
AuthorizationType: NONE
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${requestHandler.Arn}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceTagGET
OptionsMethodTag:
Type: AWS::ApiGateway::Method
Properties:
AuthorizationType: NONE
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref tag
HttpMethod: OPTIONS
Integration:
IntegrationResponses:
-
StatusCode: 200
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
method.response.header.Access-Control-Allow-Methods: "'GET,POST,OPTIONS'"
method.response.header.Access-Control-Allow-Origin: "'*'"
ResponseTemplates:
application/json: ''
PassthroughBehavior: NEVER
RequestTemplates:
application/json: '{"statusCode": 200}'
Type: MOCK
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: true
method.response.header.Access-Control-Allow-Methods: true
method.response.header.Access-Control-Allow-Origin: true
DependsOn:
- ProxyResourceTagPOST
ProxyResourceTagLinkGET:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref taglink
HttpMethod: GET
AuthorizationType: NONE
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${requestHandler.Arn}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceTagPOST
ProxyResourceTagLinkPOST:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref taglink
HttpMethod: POST
AuthorizationType: NONE
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${requestHandler.Arn}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceTagLinkGET
OptionsMethodTagLink:
Type: AWS::ApiGateway::Method
Properties:
AuthorizationType: NONE
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref taglink
HttpMethod: OPTIONS
Integration:
IntegrationResponses:
-
StatusCode: 200
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
method.response.header.Access-Control-Allow-Methods: "'GET,POST,OPTIONS'"
method.response.header.Access-Control-Allow-Origin: "'*'"
ResponseTemplates:
application/json: ''
PassthroughBehavior: NEVER
RequestTemplates:
application/json: '{"statusCode": 200}'
Type: MOCK
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: true
method.response.header.Access-Control-Allow-Methods: true
method.response.header.Access-Control-Allow-Origin: true
DependsOn:
- ProxyResourceTagLinkPOST
Deployment:
DependsOn: "OptionsMethodTagLink"
Type: AWS::ApiGateway::Deployment
Properties:
RestApiId:
Ref: "ApplicationAPI"
Description: "Application API deployment"
StageName: "prod"
Step 6: Update the Outputs in the CloudFormation Template
Update the Outputs to include each endpoint URL, as well as the base URL for the API.
Outputs:
APIBaseUrl:
Description: Application API Gateway base URL
Value: !Join
- ''
- - 'https://'
- !Ref ApplicationAPI
- '.execute-api.'
- !Ref AWS::Region
- '.amazonaws.com/prod'
APILinkUrl:
Description: Application API Gateway link endpoint URL
Value: !Join
- ''
- - 'https://'
- !Ref ApplicationAPI
- '.execute-api.'
- !Ref AWS::Region
- '.amazonaws.com/prod/link'
APITagUrl:
Description: Application API Gateway tag endpoint URL
Value: !Join
- ''
- - 'https://'
- !Ref ApplicationAPI
- '.execute-api.'
- !Ref AWS::Region
- '.amazonaws.com/prod/tag'
APITagLinkUrl:
Description: Application API Gateway taglink endpoint URL
Value: !Join
- ''
- - 'https://'
- !Ref ApplicationAPI
- '.execute-api.'
- !Ref AWS::Region
- '.amazonaws.com/prod/taglink'
Step 7: Launch the CloudFormation Template
After an update to the Description, the CloudFormation template should now look something like the below:
AWSTemplateFormatVersion: "2010-09-09"
Description: Creates a simple API with three endpoints that can be called to POST/GET links and tags and tag links, with a DynamoDB database
Resources:
linkTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
-
AttributeName: "link_id"
AttributeType: "S"
-
AttributeName: "link"
AttributeType: "S"
KeySchema:
-
AttributeName: "link_id"
KeyType: "HASH"
-
AttributeName: "link"
KeyType: "RANGE"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
TableName: "linkTable"
GlobalSecondaryIndexes:
-
IndexName: "link_index"
KeySchema:
-
AttributeName: "link"
KeyType: "HASH"
-
AttributeName: "link_id"
KeyType: "RANGE"
Projection:
ProjectionType: "KEYS_ONLY"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
tagTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
-
AttributeName: "tag_id"
AttributeType: "S"
-
AttributeName: "tag"
AttributeType: "S"
KeySchema:
-
AttributeName: "tag_id"
KeyType: "HASH"
-
AttributeName: "tag"
KeyType: "RANGE"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
TableName: "tagTable"
GlobalSecondaryIndexes:
-
IndexName: "tag_index"
KeySchema:
-
AttributeName: "tag"
KeyType: "HASH"
-
AttributeName: "tag_id"
KeyType: "RANGE"
Projection:
ProjectionType: "KEYS_ONLY"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
tagLinkTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
-
AttributeName: "tag_id"
AttributeType: "S"
-
AttributeName: "link_id"
AttributeType: "S"
KeySchema:
-
AttributeName: "tag_id"
KeyType: "HASH"
-
AttributeName: "link_id"
KeyType: "RANGE"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
TableName: "tagLinkTable"
requestHandlerLambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Principal:
Service:
- "lambda.amazonaws.com"
Action:
- "sts:AssumeRole"
Path: "/"
Policies:
-
PolicyName: "root"
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action:
- logs:CreateLogGroup
- logs:PutLogEvents
- logs:CreateLogStream
Resource: "*"
-
PolicyName: "db_access"
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action:
- dynamodb:DeleteItem
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:Scan
- dynamodb:UpdateItem
- dynamodb:Query
Resource: !Sub "arn:aws:dynamodb:${AWS::Region}:${AWS::AccountId}:table/*"
RoleName: requestHandlerLambdaExecutionRole
requestHandler:
Type: "AWS::Lambda::Function"
Properties:
Code:
ZipFile: !Sub |
import json
import boto3
import uuid
# Establish credentials
session_var = boto3.session.Session()
credentials = session_var.get_credentials()
""" --- Parameters --- """
# DynamoDB tables, read lambda function environment variables or initialize with defaults if they not exist.
DYNAMODB_LINK_TABLE = 'linkTable'
DYNAMODB_TAG_TABLE = 'tagTable'
DYNAMODB_TAG_LINK_TABLE = 'tagLinkTable'
# Initialize DynamoDB Client
dynamodb = boto3.client('dynamodb')
def store_link(link):
link_id = str(uuid.uuid4())
dynamodb.put_item(
TableName=DYNAMODB_LINK_TABLE,
Item={
'link_id': {
'S': str(link_id)
},
'link': {
'S': str(link)
}
}
)
return link_id
def store_tag(tag):
tag_id = str(uuid.uuid4())
dynamodb.put_item(
TableName=DYNAMODB_TAG_TABLE,
Item={
'tag_id': {
'S': str(tag_id)
},
'tag': {
'S': str(tag)
}
}
)
return tag_id
def store_taglink(tag_id, link_id):
dynamodb.put_item(
TableName=DYNAMODB_TAG_LINK_TABLE,
Item={
'tag_id': {
'S': str(tag_id)
},
'link_id': {
'S': str(link_id)
}
}
)
def get_link_id(link):
result = dynamodb.query(
TableName=DYNAMODB_LINK_TABLE,
IndexName='link_index',
KeyConditionExpression='link = :link',
ExpressionAttributeValues={
':link': {
"S": link
}
}
)
if result['Count'] > 0:
link_id = result['Items'][0]['tag_id']['S']
else:
link_id = store_link(link)
return link_id
def get_tag_id(tag):
result = dynamodb.query(
TableName=DYNAMODB_TAG_TABLE,
IndexName='tag_index',
KeyConditionExpression='tag = :tag',
ExpressionAttributeValues={
':tag': {
"S": tag
}
}
)
if result['Count'] > 0:
tag_id = result['Items'][0]['tag_id']['S']
else:
tag_id = store_tag(tag)
return tag_id
def get_tags(tag_id=None):
tags = []
if tag_id is None:
result = dynamodb.scan(
TableName=DYNAMODB_TAG_TABLE,
Select="SPECIFIC_ATTRIBUTES",
ProjectionExpression="tag_id,tag"
)
else:
# Run a query with a filter on tag_id on the tagTable
result = dynamodb.query(
TableName=DYNAMODB_TAG_TABLE,
KeyConditionExpression='tag_id = :tag_id',
ExpressionAttributeValues={
':tag_id': {
"S": tag_id
}
}
)
if result['Count'] > 0:
for x in range(result['Count']):
tag = {}
tag['tag_id'] = result['Items'][x]['tag_id']['S']
tag['tag'] = result['Items'][x]['tag']['S']
tags.append(tag)
return tags
def extract_link_result(result):
links = []
if result['Count'] > 0:
for x in range(result['Count']):
link = {}
link['link_id'] = result['Items'][x]['link_id']['S']
link['link'] = result['Items'][x]['link']['S']
links.append(link)
return links
def get_links(tag_id=None, link_id=None):
links = []
link_id_list = []
if tag_id is None and link_id is None:
result = dynamodb.scan(
TableName=DYNAMODB_LINK_TABLE,
Select="SPECIFIC_ATTRIBUTES",
ProjectionExpression="link_id,link"
)
links = extract_link_result(result)
elif link_id is not None:
# Run a query with a filter on link_id on the linkTable
result = dynamodb.query(
TableName=DYNAMODB_LINK_TABLE,
KeyConditionExpression='link_id = :link_id',
ExpressionAttributeValues={
':link_id': {
"S": link_id
}
}
)
links = extract_link_result(result)
else:
# Run a query with a filter on tag_id on the tagLinkTable
result = dynamodb.query(
TableName=DYNAMODB_TAG_LINK_TABLE,
KeyConditionExpression='tag_id = :tag_id',
ExpressionAttributeValues={
':tag_id': {
"S": tag_id
}
}
)
if result['Count'] > 0:
for x in range(result['Count']):
link_id_list.append(result['Items'][x]['link_id']['S'])
# From the retrieved link_ids, retrieve the links from the link table
if len(link_id_list) > 0:
for link_id in link_id_list:
result = dynamodb.query(
TableName=DYNAMODB_LINK_TABLE,
KeyConditionExpression='link_id = :link_id',
ExpressionAttributeValues={
':link_id': {
"S": link_id
}
}
)
if result['Count'] > 0:
link = {}
link['link_id'] = link_id
# There should only be one result per link_id
link['link'] = result['Items'][0]['link']['S']
links.append(link)
return links
def error_response(error_message, aws_request_id):
return {
"statusCode": 500,
"isBase64Encoded": False,
"body": json.dumps({
"Error": error_message,
"Reference": aws_request_id,
}),
"headers": {
'Access-Control-Allow-Origin': '*',
},
}
def bad_request_response(error_message, aws_request_id):
return {
"statusCode": 400,
"isBase64Encoded": False,
"body": json.dumps({
"Error": error_message,
"Reference": aws_request_id,
}),
"headers": {
'Access-Control-Allow-Origin': '*',
},
}
def lambda_handler(event, context):
try:
print(event)
print(context)
return_body = ""
if event['resource'] == '/link':
if event['httpMethod'] == "POST":
post_body = json.loads(event['body'])
link = post_body['link']
tag = post_body.get('tag')
tag_id = post_body.get('tag_id')
if tag_id is None:
tag_id = get_tag_id(tag)
else:
tags = get_tags(tag_id)
if not tags:
return bad_request_response('Unknown tag_id ' + tag_id, context.aws_request_id)
link_id = get_link_id(link)
store_taglink(tag_id, link_id)
return_body = json.dumps({"link_id": link_id})
elif event['httpMethod'] == "GET":
tag_id = None
link_id = None
query_params = event['queryStringParameters']
if query_params is not None:
# query_params = json.loads(query_params)
tag_id = query_params.get('tag_id', None)
tag = query_params.get('tag', None)
link_id = query_params.get('link_id', None)
if tag_id is None and tag is not None:
tag_id = get_tag_id(tag)
links = get_links(tag_id, link_id)
return_body = json.dumps(links)
elif event['resource'] == '/tag':
if event['httpMethod'] == "POST":
post_body = json.loads(event['body'])
tag = post_body.get('tag')
tag_id = store_tag(tag)
return_body = json.dumps({"tag_id": tag_id})
elif event['httpMethod'] == "GET":
tag_id = None
query_params = event['queryStringParameters']
if query_params is not None:
tag_id = query_params.get('tag_id', None)
tag = query_params.get('tag', None)
if tag_id is None and tag is not None:
tag_id = get_tag_id(tag)
tags = get_tags(tag_id)
return_body = json.dumps(tags)
elif event['resource'] == '/taglink':
if event['httpMethod'] == "POST":
post_body = json.loads(event['body'])
tag_id = post_body.get('tag_id')
link_id = post_body.get('link_id')
tags = get_tags(tag_id)
links = get_links(tag_id=None, link_id=link_id)
if not tags:
return bad_request_response('Unknown tag_id ' + tag_id, context.aws_request_id)
if not links:
return bad_request_response('Unknown link_id ' + link_id, context.aws_request_id)
store_taglink(tag_id, link_id)
return_body = json.dumps({"tag_id": tag_id, "link_id": link_id})
# Because this Lambda function is called by an API Gateway proxy integration
# the result object must use the following structure.
return {
"statusCode": 200,
"isBase64Encoded": False,
"body": return_body,
"headers": {
'Access-Control-Allow-Origin': '*',
},
}
except Exception as err:
print(str(err))
# If there is an error during processing, catch it and return
# from the Lambda function successfully. Specify a 500 HTTP status
# code and provide an error message in the body. This will provide a
# more meaningful error response to the end client.
return error_response(str(err), context.aws_request_id)
Description: Lambda function for processing api endpoint requests
FunctionName: requestHandler
Handler: index.lambda_handler
Role:
Fn::GetAtt:
- "requestHandlerLambdaExecutionRole"
- "Arn"
Runtime: python3.8
DependsOn:
- requestHandlerLambdaExecutionRole
ApplicationAPI:
Type: AWS::ApiGateway::RestApi
Properties:
Name: ApplicationAPI
Description: Simple API for an application
DependsOn:
- requestHandler
LambdaPermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !GetAtt
- requestHandler
- Arn
Action: 'lambda:InvokeFunction'
Principal: apigateway.amazonaws.com
SourceArn: !Join ["", [!Sub "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:", !Ref ApplicationAPI, "/*"]]
DependsOn:
- ApplicationAPI
link:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref ApplicationAPI
ParentId: !GetAtt [ApplicationAPI, RootResourceId]
PathPart: 'link'
DependsOn:
- ApplicationAPI
tag:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref ApplicationAPI
ParentId: !GetAtt [ApplicationAPI, RootResourceId]
PathPart: 'tag'
DependsOn:
- ApplicationAPI
taglink:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref ApplicationAPI
ParentId: !GetAtt [ApplicationAPI, RootResourceId]
PathPart: 'taglink'
DependsOn:
- ApplicationAPI
ProxyResourceLinkGET:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref link
HttpMethod: GET
AuthorizationType: NONE
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${requestHandler.Arn}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ApplicationAPI
ProxyResourceLinkPOST:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref link
HttpMethod: POST
AuthorizationType: NONE
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${requestHandler.Arn}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceLinkGET
OptionsMethodLink:
Type: AWS::ApiGateway::Method
Properties:
AuthorizationType: NONE
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref link
HttpMethod: OPTIONS
Integration:
IntegrationResponses:
-
StatusCode: 200
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
method.response.header.Access-Control-Allow-Methods: "'GET,POST,OPTIONS'"
method.response.header.Access-Control-Allow-Origin: "'*'"
ResponseTemplates:
application/json: ''
PassthroughBehavior: NEVER
RequestTemplates:
application/json: '{"statusCode": 200}'
Type: MOCK
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: true
method.response.header.Access-Control-Allow-Methods: true
method.response.header.Access-Control-Allow-Origin: true
DependsOn:
- ProxyResourceLinkPOST
ProxyResourceTagGET:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref tag
HttpMethod: GET
AuthorizationType: NONE
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${requestHandler.Arn}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceLinkPOST
ProxyResourceTagPOST:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref tag
HttpMethod: POST
AuthorizationType: NONE
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${requestHandler.Arn}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceTagGET
OptionsMethodTag:
Type: AWS::ApiGateway::Method
Properties:
AuthorizationType: NONE
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref tag
HttpMethod: OPTIONS
Integration:
IntegrationResponses:
-
StatusCode: 200
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
method.response.header.Access-Control-Allow-Methods: "'GET,POST,OPTIONS'"
method.response.header.Access-Control-Allow-Origin: "'*'"
ResponseTemplates:
application/json: ''
PassthroughBehavior: NEVER
RequestTemplates:
application/json: '{"statusCode": 200}'
Type: MOCK
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: true
method.response.header.Access-Control-Allow-Methods: true
method.response.header.Access-Control-Allow-Origin: true
DependsOn:
- ProxyResourceTagPOST
ProxyResourceTagLinkGET:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref taglink
HttpMethod: GET
AuthorizationType: NONE
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${requestHandler.Arn}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceTagPOST
ProxyResourceTagLinkPOST:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref taglink
HttpMethod: POST
AuthorizationType: NONE
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${requestHandler.Arn}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceTagLinkGET
OptionsMethodTagLink:
Type: AWS::ApiGateway::Method
Properties:
AuthorizationType: NONE
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref taglink
HttpMethod: OPTIONS
Integration:
IntegrationResponses:
-
StatusCode: 200
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
method.response.header.Access-Control-Allow-Methods: "'GET,POST,OPTIONS'"
method.response.header.Access-Control-Allow-Origin: "'*'"
ResponseTemplates:
application/json: ''
PassthroughBehavior: NEVER
RequestTemplates:
application/json: '{"statusCode": 200}'
Type: MOCK
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: true
method.response.header.Access-Control-Allow-Methods: true
method.response.header.Access-Control-Allow-Origin: true
DependsOn:
- ProxyResourceTagLinkPOST
Deployment:
DependsOn: "OptionsMethodTagLink"
Type: AWS::ApiGateway::Deployment
Properties:
RestApiId:
Ref: "ApplicationAPI"
Description: "Application API deployment"
StageName: "prod"
Outputs:
APIBaseUrl:
Description: Application API Gateway base URL
Value: !Join
- ''
- - 'https://'
- !Ref ApplicationAPI
- '.execute-api.'
- !Ref AWS::Region
- '.amazonaws.com/prod'
APILinkUrl:
Description: Application API Gateway link endpoint URL
Value: !Join
- ''
- - 'https://'
- !Ref ApplicationAPI
- '.execute-api.'
- !Ref AWS::Region
- '.amazonaws.com/prod/link'
APITagUrl:
Description: Application API Gateway tag endpoint URL
Value: !Join
- ''
- - 'https://'
- !Ref ApplicationAPI
- '.execute-api.'
- !Ref AWS::Region
- '.amazonaws.com/prod/tag'
APITagLinkUrl:
Description: Application API Gateway taglink endpoint URL
Value: !Join
- ''
- - 'https://'
- !Ref ApplicationAPI
- '.execute-api.'
- !Ref AWS::Region
- '.amazonaws.com/prod/taglink'
Save the CFT to a file called setupapidb2.yml and launch it with the following command:
$ aws cloudformation create-stack --stack-name apidb2 --template-body file://setupapidb2.yml --capabilities CAPABILITY_NAMED_IAM
Poll the progress of the CloudFormation stack with the following command:
$ aws cloudformation describe-stacks --stack-name apidb2
Once complete, the value of “StackStatus” in the response should be “CREATE_COMPLETE” and you should have four API URLs in the “Outputs” section, under each “OutputValue”. The links should be of the form: “https://xxxxxxxxxx.execute-api.your-region.amazonaws.com/prod/ENDPOINT” for each of the endpoints link, tag and taglink. There should also be a base URL under “APIBaseUrl”. This is the URL of the API, minus any specific endpoint.
Step 8: Test the API
Test that everything is working by using the bash test script below. You may need to install curl and jq first with sudo yum install epel-release -y; sudo yum install curl jq or similar, depending on your operating system. The jq package is useful for parsing and displaying json. Once both are installed, copy the script below to a file called test.sh. Be sure to give it executable permissions (sudo chmod u+x test.sh for linux). Then run it with the API Base URL as the sole argument, e.g:
$ /usr/bin/bash test.sh https://xxxxxxxxxx.execute-api.your-region.amazonaws.com/prod
The test script is below:
BASEURL=$1
echo "=========================="
echo "Test POST /link link=https://www.allthecoding.com, tag=blog"
resp=$(curl -X "POST" -H "Content-Type: application/json" -d "{\"link\": \"https://www.allthecoding.com\", \"tag\": \"blog\"}" $BASEURL/link)
echo $resp
link_id1=$(echo $resp | jq '.link_id' | tr -d '"')
echo $link_id1
echo "=========================="
echo "Test GET /tag tag=blog"
resp=$(curl -X "GET" -H "Content-Type: application/json" $BASEURL/tag?tag=blog)
echo $resp
tag_id1=$(echo $resp | jq '.[0].tag_id' | tr -d '"')
echo $tag_id1
echo "=========================="
echo "Test POST /tag tag=stuff"
resp=$(curl -X "POST" -H "Content-Type: application/json" -d "{\"tag\": \"stuff\"}" $BASEURL/tag)
echo $resp
tag_id2=$(echo $resp | jq '.tag_id' | tr -d '"')
echo $tag_id2
echo "=========================="
echo "Test POST /link link=https://www.google.com, tag=stuff"
resp=$(curl -X "POST" -H "Content-Type: application/json" -d "{\"link\": \"https://www.google.com\", \"tag\": \"stuff\"}" $BASEURL/link)
echo $resp
link_id2=$(echo $resp | jq '.link_id' | tr -d '"')
echo $link_id2
echo "=========================="
echo "Test POST /link link=https://www.google.com, tag_id=$tag_id1"
resp=$(curl -X "POST" -H "Content-Type: application/json" -d "{\"link\": \"https://www.google.com\", \"tag_id\": \"$tag_id1\"}" $BASEURL/link)
echo $resp
link_id2=$(echo $resp | jq '.link_id' | tr -d '"')
echo $link_id2
echo "=========================="
echo "Test POST /link link=https://www.google.com, tag_id=invalid. Expect 400 response: Unknown tag_id invalid"
resp=$(curl -X "POST" -H "Content-Type: application/json" -d "{\"link\": \"https://www.google.com\", \"tag_id\": \"invalid\"}" $BASEURL/link)
echo $resp
link_id2=$(echo $resp | jq '.link_id' | tr -d '"')
echo $link_id2
echo "=========================="
echo "Test GET /link"
resp=$(curl -X "GET" -H "Content-Type: application/json" $BASEURL/link)
echo $resp
echo "=========================="
echo "Test GET /link tag_id=$tag_id1"
resp=$(curl -X "GET" -H "Content-Type: application/json" $BASEURL/link?tag_id=$tag_id1)
echo $resp
echo "=========================="
echo "Test GET /link tag=blog"
resp=$(curl -X "GET" -H "Content-Type: application/json" $BASEURL/link?tag=blog)
echo $resp
echo "=========================="
echo "Test GET /link link_id=$link_id1"
resp=$(curl -X "GET" -H "Content-Type: application/json" $BASEURL/link?link_id=$link_id1)
echo $resp
echo "=========================="
echo "Test GET /tag tag_id=$tag_id1"
resp=$(curl -X "GET" -H "Content-Type: application/json" $BASEURL/tag?tag_id=$tag_id1)
echo $resp
echo "=========================="
echo "Test GET /tag"
resp=$(curl -X "GET" -H "Content-Type: application/json" $BASEURL/tag)
echo $resp
echo "=========================="
echo "Test POST /taglink tag_id=$tag_id1, link_id=$link_id1"
resp=$(curl -X "POST" -H "Content-Type: application/json" -d "{\"link_id\": \"$link_id1\", \"tag_id\": \"$tag_id1\"}" $BASEURL/taglink)
echo $resp
echo "=========================="
echo "Test POST /taglink tag_id=invalid, link_id=$link_id1. Expect 400 response: Unknown tag_id invalid"
resp=$(curl -X "POST" -H "Content-Type: application/json" -d "{\"link_id\": \"$link_id1\", \"tag_id\": \"invalid\"}" $BASEURL/taglink)
echo $resp
echo "=========================="
echo "Test POST /taglink tag_id=$tag_id1, link_id=invalid. Expect 400 response: Unknown link_id invalid"
resp=$(curl -X "POST" -H "Content-Type: application/json" -d "{\"link_id\": \"invalid\", \"tag_id\": \"$tag_id1\"}" $BASEURL/taglink)
echo $resp
If all goes well, the script should run with no errors apart from the three expected error responses, where the script will print the expected error response before making the request.
Step 9: Clean Up
If you don’t want to keep your API (note that AWS will charge you for it if you don’t have Free Tier on your account), delete the CFT stack (all resources created) by running:
$ aws cloudformation delete-stack --stack-name apidb2
You will then need to remove any logs that were generated by the Lambda function. To do this:
- Log into the AWS Console (your AWS account) using your Administrator IAM user, and navigate to CloudWatch.
- On the left-hand side, select Logs -> Log Groups.
- Select ‘aws/lambda/requestHandler’.
- Under Actions, select ‘Delete log groups’ and confirm.
Conclusion
You should now be able to create a simple API on AWS with a CloudFormation template, that has three endpoints and three DynamoDB database tables. You’ve also learned how to add Global Secondary Indexes for filter queries. If you want to find out how to add AWS Cognito authentication to the API, read my follow-on post, Creating an API with AWS: Part 4: Cognito Authentication.
Thanks for reading!
2 Responses
[…] Creating an API with AWS: Part 3: Additional Endpoints December 4, 2021 […]
[…] Creating an API with AWS: Part 3: Additional Endpoints December 4, 2021 […]