In my post Creating an API with AWS: Part 5: DELETE methods, I added DELETE methods to each endpoint, to give us full CRUD (Create, Read, Update, Delete) functionality. In this post, I’ll show you how to break up the (rather cumbersome now) CloudFormation template into several nested CloudFormation templates, to make them easier to manage.
Prerequisites
Before you start, you’ll need the following things:
- An AWS account
- An Administrator IAM User with which to use the AWS CLI
- AWS CLI set up on your local machine
Note that AWS will charge you for creating and hosting this API if you don’t have Free Tier on your account.
Step 1: Create an S3 Bucket
To be able to use nested CloudFormation templates, it is necessary to create an s3 bucket with AWS to store the nested templates. Choose a unique bucket name and replace ‘<YOUR_UNIQUE_BUCKET_NAME>’ below, before running:
$ export $BUCKET=<YOUR_UNIQUE_BUCKET_NAME> $ aws s3api create-bucket --bucket $BUCKET
Step 2: Create a New CloudFormation Parent Template
Create a new CloudFormation template called nested.yml. This will be the parent CloudFormation template, that will launch, in turn, the child CloudFormation templates. The contents should be as below:
AWSTemplateFormatVersion: 2010-09-09
Description: Creates a simple API with three endpoints that can be called to POST/GET/DELETE links and tags and tag links, with a DynamoDB database
Parameters:
BucketName:
Description: Name of the s3 bucket
Type: String
Resources:
DatabaseStack:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Join
- ''
- - 'https://'
- !Ref BucketName
- '.s3.amazonaws.com'
- '/database.yml'
CognitoStack:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Join
- ''
- - 'https://'
- !Ref BucketName
- '.s3.amazonaws.com'
- '/cognito.yml'
LambdaStack:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Join
- ''
- - 'https://'
- !Ref BucketName
- '.s3.amazonaws.com'
- '/lambda.yml'
APIStack:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: !Join
- ''
- - 'https://'
- !Ref BucketName
- '.s3.amazonaws.com'
- '/api.yml'
Parameters:
LambdaARN: !GetAtt LambdaStack.Outputs.LambdaARN
UserPoolARN: !GetAtt CognitoStack.Outputs.UserPoolARN
DependsOn:
- CognitoStack
- LambdaStack
Outputs:
APIBaseUrl:
Description: Application API Gateway base URL
Value: !GetAtt APIStack.Outputs.APIBaseUrl
APILinkUrl:
Description: Application API Gateway link endpoint URL
Value: !GetAtt APIStack.Outputs.APILinkUrl
APITagUrl:
Description: Application API Gateway tag endpoint URL
Value: !GetAtt APIStack.Outputs.APITagUrl
APITagLinkUrl:
Description: Application API Gateway taglink endpoint URL
Value: !GetAtt APIStack.Outputs.APITagLinkUrl
UserPoolID:
Description: The Cognito user pool ID
Value: !GetAtt CognitoStack.Outputs.UserPoolID
APIClientID:
Description: The API user pool client ID
Value: !GetAtt CognitoStack.Outputs.APIClientID
Note that instead of creating individual resources such as a Lambda, APIGateway etc, this template creates other CloudFormation stacks as resources, and they will create the individual resources for each stack. This means we can organise and separate different groups of resources into different CloudFormation templates, making them a bit easier to manage. The outputs for the parent stack are derived from the outputs of each nested stack.
Step 3: Create the Database CloudFormation Template
Create a CFT file called database.yml which contains the database resources as below:
AWSTemplateFormatVersion: "2010-09-09"
Description: Creates three DynamoDB database tables - linkTable, tagTable, tagLinkTable
Resources:
linkTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
-
AttributeName: "link_id"
AttributeType: "S"
-
AttributeName: "link"
AttributeType: "S"
KeySchema:
-
AttributeName: "link_id"
KeyType: "HASH"
-
AttributeName: "link"
KeyType: "RANGE"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
TableName: "linkTable"
GlobalSecondaryIndexes:
-
IndexName: "link_index"
KeySchema:
-
AttributeName: "link"
KeyType: "HASH"
-
AttributeName: "link_id"
KeyType: "RANGE"
Projection:
ProjectionType: "KEYS_ONLY"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
tagTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
-
AttributeName: "tag_id"
AttributeType: "S"
-
AttributeName: "tag"
AttributeType: "S"
KeySchema:
-
AttributeName: "tag_id"
KeyType: "HASH"
-
AttributeName: "tag"
KeyType: "RANGE"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
TableName: "tagTable"
GlobalSecondaryIndexes:
-
IndexName: "tag_index"
KeySchema:
-
AttributeName: "tag"
KeyType: "HASH"
-
AttributeName: "tag_id"
KeyType: "RANGE"
Projection:
ProjectionType: "KEYS_ONLY"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
tagLinkTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
-
AttributeName: "tag_id"
AttributeType: "S"
-
AttributeName: "link_id"
AttributeType: "S"
KeySchema:
-
AttributeName: "tag_id"
KeyType: "HASH"
-
AttributeName: "link_id"
KeyType: "RANGE"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
TableName: "tagLinkTable"
GlobalSecondaryIndexes:
-
IndexName: "link_id_index"
KeySchema:
-
AttributeName: "link_id"
KeyType: "HASH"
-
AttributeName: "tag_id"
KeyType: "RANGE"
Projection:
ProjectionType: "KEYS_ONLY"
ProvisionedThroughput:
ReadCapacityUnits: "5"
WriteCapacityUnits: "5"
Step 4: Create the Cognito CloudFormation Template
Create a CFT file called cognito.yml which contains the Cognito resources as below:
AWSTemplateFormatVersion: "2010-09-09"
Description: Creates a Cognito User Pool and a Cognito User Pool Client
Resources:
apiuserpool:
Type: AWS::Cognito::UserPool
Properties:
AdminCreateUserConfig:
AllowAdminCreateUserOnly: true
AutoVerifiedAttributes:
- email
UsernameAttributes:
- email
UsernameConfiguration:
CaseSensitive: false
UserPoolName: apiuserpool
applicationAPIUserPoolClient:
Type: AWS::Cognito::UserPoolClient
Properties:
AccessTokenValidity: 1
ClientName: applicationapiclient
EnableTokenRevocation: true
ExplicitAuthFlows:
- ALLOW_USER_PASSWORD_AUTH
- ALLOW_REFRESH_TOKEN_AUTH
- ALLOW_ADMIN_USER_PASSWORD_AUTH
GenerateSecret: true
IdTokenValidity: 1
PreventUserExistenceErrors: ENABLED
RefreshTokenValidity: 1
SupportedIdentityProviders:
- COGNITO
UserPoolId: !Ref apiuserpool
DependsOn:
- apiuserpool
Outputs:
UserPoolID:
Description: The Cognito user pool ID
Value: !Ref apiuserpool
UserPoolARN:
Description: The Cognito user pool ARN
Value: !GetAtt apiuserpool.Arn
APIClientID:
Description: The API user pool client ID
Value: !Ref applicationAPIUserPoolClient
Step 5: Create the Lambda CloudFormation Template
Create a CFT file called lambda.yml which contains the Lambda function resources as below:
AWSTemplateFormatVersion: "2010-09-09"
Description: Creates a Lambda Function and Lambda Execution Role
Resources:
requestHandlerLambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: Allow
Principal:
Service:
- "lambda.amazonaws.com"
Action:
- "sts:AssumeRole"
Path: "/"
Policies:
-
PolicyName: "root"
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action:
- logs:CreateLogGroup
- logs:PutLogEvents
- logs:CreateLogStream
Resource: "*"
-
PolicyName: "db_access"
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action:
- dynamodb:DeleteItem
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:Scan
- dynamodb:UpdateItem
- dynamodb:Query
Resource: !Sub "arn:aws:dynamodb:${AWS::Region}:${AWS::AccountId}:table/*"
RoleName: requestHandlerLambdaExecutionRole
requestHandler:
Type: "AWS::Lambda::Function"
Properties:
Code:
ZipFile: !Sub |
import json
import boto3
import uuid
from botocore.exceptions import ClientError
# Establish credentials
session_var = boto3.session.Session()
credentials = session_var.get_credentials()
""" --- Parameters --- """
# DynamoDB tables, read lambda function environment variables or initialize with defaults if they not exist.
DYNAMODB_LINK_TABLE = 'linkTable'
DYNAMODB_TAG_TABLE = 'tagTable'
DYNAMODB_TAG_LINK_TABLE = 'tagLinkTable'
# Initialize DynamoDB Client
dynamodb = boto3.client('dynamodb')
def store_link(link):
link_id = str(uuid.uuid4())
dynamodb.put_item(
TableName=DYNAMODB_LINK_TABLE,
Item={
'link_id': {
'S': str(link_id)
},
'link': {
'S': str(link)
}
}
)
return link_id
def store_tag(tag):
tag_id = str(uuid.uuid4())
dynamodb.put_item(
TableName=DYNAMODB_TAG_TABLE,
Item={
'tag_id': {
'S': str(tag_id)
},
'tag': {
'S': str(tag)
}
}
)
return tag_id
def store_taglink(tag_id, link_id):
dynamodb.put_item(
TableName=DYNAMODB_TAG_LINK_TABLE,
Item={
'tag_id': {
'S': str(tag_id)
},
'link_id': {
'S': str(link_id)
}
}
)
def get_link_id(link):
result = dynamodb.query(
TableName=DYNAMODB_LINK_TABLE,
IndexName='link_index',
KeyConditionExpression='link = :link',
ExpressionAttributeValues={
':link': {
"S": link
}
}
)
if result['Count'] > 0:
link_id = result['Items'][0]['link_id']['S']
else:
link_id = store_link(link)
return link_id
def get_tag_id(tag):
result = dynamodb.query(
TableName=DYNAMODB_TAG_TABLE,
IndexName='tag_index',
KeyConditionExpression='tag = :tag',
ExpressionAttributeValues={
':tag': {
"S": tag
}
}
)
if result['Count'] > 0:
tag_id = result['Items'][0]['tag_id']['S']
else:
tag_id = store_tag(tag)
return tag_id
def get_tags(tag_id=None):
tags = []
if tag_id is None:
result = dynamodb.scan(
TableName=DYNAMODB_TAG_TABLE,
Select="SPECIFIC_ATTRIBUTES",
ProjectionExpression="tag_id,tag"
)
else:
# Run a query with a filter on tag_id on the tagTable
# Get IDs for each keyword
result = dynamodb.query(
TableName=DYNAMODB_TAG_TABLE,
KeyConditionExpression='tag_id = :tag_id',
ExpressionAttributeValues={
':tag_id': {
"S": tag_id
}
}
)
if result['Count'] > 0:
for x in range(result['Count']):
tag = {}
tag['tag_id'] = result['Items'][x]['tag_id']['S']
tag['tag'] = result['Items'][x]['tag']['S']
tags.append(tag)
return tags
def extract_link_result(result):
links = []
if result['Count'] > 0:
for x in range(result['Count']):
link = {}
link['link_id'] = result['Items'][x]['link_id']['S']
link['link'] = result['Items'][x]['link']['S']
links.append(link)
return links
def get_links(tag_id=None, link_id=None):
links = []
link_id_list = []
if tag_id is None and link_id is None:
result = dynamodb.scan(
TableName=DYNAMODB_LINK_TABLE,
Select="SPECIFIC_ATTRIBUTES",
ProjectionExpression="link_id,link"
)
links = extract_link_result(result)
elif link_id is not None:
# Run a query with a filter on link_id on the linkTable
result = dynamodb.query(
TableName=DYNAMODB_LINK_TABLE,
KeyConditionExpression='link_id = :link_id',
ExpressionAttributeValues={
':link_id': {
"S": link_id
}
}
)
links = extract_link_result(result)
else:
# Run a query with a filter on tag_id on the tagLinkTable
result = dynamodb.query(
TableName=DYNAMODB_TAG_LINK_TABLE,
KeyConditionExpression='tag_id = :tag_id',
ExpressionAttributeValues={
':tag_id': {
"S": tag_id
}
}
)
if result['Count'] > 0:
for x in range(result['Count']):
link_id_list.append(result['Items'][x]['link_id']['S'])
# From the retrieved link_ids, retrieve the links from the link table
if len(link_id_list) > 0:
for link_id in link_id_list:
result = dynamodb.query(
TableName=DYNAMODB_LINK_TABLE,
KeyConditionExpression='link_id = :link_id',
ExpressionAttributeValues={
':link_id': {
"S": link_id
}
}
)
if result['Count'] > 0:
link = {}
link['link_id'] = link_id
# There should only be one result per link_id
link['link'] = result['Items'][0]['link']['S']
links.append(link)
return links
def get_taglinks(tag_id=None, link_id=None):
taglinks = []
if tag_id is not None and link_id is None:
# Run a query with a filter on tag_id on the tagLinkTable
result = dynamodb.query(
TableName=DYNAMODB_TAG_LINK_TABLE,
KeyConditionExpression='tag_id = :tag_id',
ExpressionAttributeValues={
':tag_id': {
"S": tag_id
}
}
)
if result['Count'] > 0:
for x in range(result['Count']):
taglink = {}
taglink['tag_id'] = tag_id
taglink['link_id'] = result['Items'][x]['link_id']['S']
taglinks.append(taglink)
elif link_id is not None and tag_id is None:
# Run a query with a filter on link_id on the tagLinkTable
result = dynamodb.query(
TableName=DYNAMODB_TAG_LINK_TABLE,
IndexName='link_id_index',
KeyConditionExpression='link_id = :link_id',
ExpressionAttributeValues={
':link_id': {
"S": link_id
}
}
)
if result['Count'] > 0:
for x in range(result['Count']):
taglink = {}
taglink['link_id'] = link_id
taglink['tag_id'] = result['Items'][x]['tag_id']['S']
taglinks.append(taglink)
elif link_id is not None and tag_id is not None:
result = dynamodb.get_item(
TableName=DYNAMODB_TAG_LINK_TABLE,
Key={
'link_id': {
"S": link_id
},
'tag_id': {
"S": tag_id
}
}
)
if result:
taglink = {}
taglink['link_id'] = result['Item']['link_id']['S']
taglink['tag_id'] = result['Item']['tag_id']['S']
taglinks.append(taglink)
return taglinks
def delete_link(link_id):
links = get_links(tag_id=None, link_id=link_id)
for link in links:
dynamodb.delete_item(
TableName=DYNAMODB_LINK_TABLE,
Key={
'link_id': {
"S": link['link_id']
},
'link': {
"S": link['link']
}
}
)
return
def delete_tag(tag_id):
tags = get_tags(tag_id=tag_id)
for tag in tags:
dynamodb.delete_item(
TableName=DYNAMODB_TAG_TABLE,
Key={
'tag_id': {
"S": tag['tag_id'],
},
'tag': {
"S": tag['tag']
}
}
)
return
def delete_taglinks(tag_id=None, link_id=None):
taglinks = get_taglinks(tag_id=tag_id, link_id=link_id)
for taglink in taglinks:
dynamodb.delete_item(
TableName=DYNAMODB_TAG_LINK_TABLE,
Key={
'tag_id': {
"S": taglink['tag_id'],
},
'link_id': {
"S": taglink['link_id']
},
}
)
return
def error_response(error_message, aws_request_id):
return {
"statusCode": 500,
"isBase64Encoded": False,
"body": json.dumps({
"Error": error_message,
"Reference": aws_request_id,
}),
"headers": {
'Access-Control-Allow-Origin': '*',
},
}
def bad_request_response(error_message, aws_request_id):
return {
"statusCode": 400,
"isBase64Encoded": False,
"body": json.dumps({
"Error": error_message,
"Reference": aws_request_id,
}),
"headers": {
'Access-Control-Allow-Origin': '*',
},
}
def lambda_handler(event, context):
try:
print(event)
print(context)
return_body = ""
if event['resource'] == '/link':
if event['httpMethod'] == "POST":
post_body = json.loads(event['body'])
link = post_body['link']
tag = post_body.get('tag')
tag_id = post_body.get('tag_id')
if tag_id is None:
tag_id = get_tag_id(tag)
else:
tags = get_tags(tag_id)
if not tags:
return bad_request_response('Unknown tag_id ' + tag_id, context.aws_request_id)
link_id = get_link_id(link)
store_taglink(tag_id, link_id)
return_body = json.dumps({"link_id": link_id})
elif event['httpMethod'] == "GET":
tag_id = None
link_id = None
query_params = event['queryStringParameters']
if query_params is not None:
tag_id = query_params.get('tag_id', None)
tag = query_params.get('tag', None)
link_id = query_params.get('link_id', None)
if tag_id is None and tag is not None:
tag_id = get_tag_id(tag)
links = get_links(tag_id, link_id)
return_body = json.dumps(links)
elif event['httpMethod'] == "DELETE":
query_params = event['queryStringParameters']
link_id = query_params.get('link_id', None)
if not link_id:
return bad_request_response('link_id is required', context.aws_request_id)
delete_link(link_id)
delete_taglinks(tag_id=None, link_id=link_id)
return_body = json.dumps({"link_id": link_id})
elif event['resource'] == '/tag':
if event['httpMethod'] == "POST":
post_body = json.loads(event['body'])
tag = post_body.get('tag')
tag_id = get_tag_id(tag)
return_body = json.dumps({"tag_id": tag_id})
elif event['httpMethod'] == "GET":
tag_id = None
query_params = event['queryStringParameters']
if query_params is not None:
tag_id = query_params.get('tag_id', None)
tag = query_params.get('tag', None)
if tag_id is None and tag is not None:
tag_id = get_tag_id(tag)
tags = get_tags(tag_id)
return_body = json.dumps(tags)
elif event['httpMethod'] == "DELETE":
query_params = event['queryStringParameters']
tag_id = query_params.get('tag_id', None)
if not tag_id:
return bad_request_response('tag_id is required', context.aws_request_id)
delete_tag(tag_id)
delete_taglinks(tag_id=tag_id, link_id=None)
return_body = json.dumps({"tag_id": tag_id})
elif event['resource'] == '/taglink':
if event['httpMethod'] == "POST":
post_body = json.loads(event['body'])
tag_id = post_body.get('tag_id')
link_id = post_body.get('link_id')
tags = get_tags(tag_id)
links = get_links(tag_id=None, link_id=link_id)
if not tags:
return bad_request_response('Unknown tag_id ' + tag_id, context.aws_request_id)
if not links:
return bad_request_response('Unknown link_id ' + link_id, context.aws_request_id)
store_taglink(tag_id, link_id)
return_body = json.dumps({"tag_id": tag_id, "link_id": link_id})
elif event['httpMethod'] == "DELETE":
query_params = event['queryStringParameters']
tag_id = query_params.get('tag_id', None)
link_id = query_params.get('link_id', None)
if not tag_id:
return bad_request_response('tag_id is required', context.aws_request_id)
if not link_id:
return bad_request_response('link_id is required', context.aws_request_id)
delete_taglinks(tag_id, link_id)
return_body = json.dumps({"tag_id": tag_id, "link_id": link_id})
# Because this Lambda function is called by an API Gateway proxy integration
# the result object must use the following structure.
return {
"statusCode": 200,
"isBase64Encoded": False,
"body": return_body,
"headers": {
'Access-Control-Allow-Origin': '*',
},
}
except Exception as err:
print(str(err))
# If there is an error during processing, catch it and return
# from the Lambda function successfully. Specify a 500 HTTP status
# code and provide an error message in the body. This will provide a
# more meaningful error response to the end client.
return error_response(str(err), context.aws_request_id)
Description: Lambda function for processing api endpoint requests
FunctionName: requestHandler
Handler: index.lambda_handler
Role:
Fn::GetAtt:
- "requestHandlerLambdaExecutionRole"
- "Arn"
Runtime: python3.8
DependsOn:
- requestHandlerLambdaExecutionRole
Outputs:
LambdaARN:
Description: The ARN of the lambda function
Value: !GetAtt requestHandler.Arn
Step 6: Create the API CloudFormation Template
Create a CFT file called api.yml which contains the API Gateway and endpoint resources as below:
AWSTemplateFormatVersion: "2010-09-09"
Description: Creates an API Gateway resource with three endpoints (link, tag, taglink) and methods GET, POST, DELETE and OPTIONS
Parameters:
LambdaARN:
Description: ARN of the Lambda Function
Type: String
UserPoolARN:
Description: ARN of the Cognito User Pool
Type: String
Resources:
ApplicationAPI:
Type: AWS::ApiGateway::RestApi
Properties:
Name: ApplicationAPI
Description: Simple API for an application
LambdaPermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !Ref LambdaARN
Action: 'lambda:InvokeFunction'
Principal: apigateway.amazonaws.com
SourceArn: !Join ["", [!Sub "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:", !Ref ApplicationAPI, "/*"]]
DependsOn:
- ApplicationAPI
link:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref ApplicationAPI
ParentId: !GetAtt [ApplicationAPI, RootResourceId]
PathPart: 'link'
DependsOn:
- ApplicationAPI
tag:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref ApplicationAPI
ParentId: !GetAtt [ApplicationAPI, RootResourceId]
PathPart: 'tag'
DependsOn:
- ApplicationAPI
taglink:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref ApplicationAPI
ParentId: !GetAtt [ApplicationAPI, RootResourceId]
PathPart: 'taglink'
DependsOn:
- ApplicationAPI
apiauth:
Type: AWS::ApiGateway::Authorizer
Properties:
IdentitySource: method.request.header.Authorization
Name: apiauth
ProviderARNs:
- !Ref UserPoolARN
RestApiId: !Ref ApplicationAPI
Type: COGNITO_USER_POOLS
ProxyResourceLinkGET:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref link
HttpMethod: GET
AuthorizationType: COGNITO_USER_POOLS
AuthorizerId: !Ref apiauth
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${LambdaARN}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ApplicationAPI
- apiauth
- link
ProxyResourceLinkPOST:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref link
HttpMethod: POST
AuthorizationType: COGNITO_USER_POOLS
AuthorizerId: !Ref apiauth
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${LambdaARN}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceLinkGET
- apiauth
ProxyResourceLinkDELETE:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref link
HttpMethod: DELETE
AuthorizationType: COGNITO_USER_POOLS
AuthorizerId: !Ref apiauth
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${LambdaARN}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceLinkPOST
- apiauth
OptionsMethodLink:
Type: AWS::ApiGateway::Method
Properties:
AuthorizationType: COGNITO_USER_POOLS
AuthorizerId: !Ref apiauth
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref link
HttpMethod: OPTIONS
Integration:
IntegrationResponses:
-
StatusCode: 200
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
method.response.header.Access-Control-Allow-Methods: "'GET,POST,OPTIONS'"
method.response.header.Access-Control-Allow-Origin: "'*'"
ResponseTemplates:
application/json: ''
PassthroughBehavior: NEVER
RequestTemplates:
application/json: '{"statusCode": 200}'
Type: MOCK
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: true
method.response.header.Access-Control-Allow-Methods: true
method.response.header.Access-Control-Allow-Origin: true
DependsOn:
- ProxyResourceLinkDELETE
- apiauth
ProxyResourceTagGET:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref tag
HttpMethod: GET
AuthorizationType: COGNITO_USER_POOLS
AuthorizerId: !Ref apiauth
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${LambdaARN}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceLinkPOST
- apiauth
ProxyResourceTagPOST:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref tag
HttpMethod: POST
AuthorizationType: COGNITO_USER_POOLS
AuthorizerId: !Ref apiauth
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${LambdaARN}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceTagGET
- apiauth
ProxyResourceTagDELETE:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref tag
HttpMethod: DELETE
AuthorizationType: COGNITO_USER_POOLS
AuthorizerId: !Ref apiauth
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${LambdaARN}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceTagPOST
- apiauth
OptionsMethodTag:
Type: AWS::ApiGateway::Method
Properties:
AuthorizationType: COGNITO_USER_POOLS
AuthorizerId: !Ref apiauth
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref tag
HttpMethod: OPTIONS
Integration:
IntegrationResponses:
-
StatusCode: 200
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
method.response.header.Access-Control-Allow-Methods: "'GET,POST,OPTIONS'"
method.response.header.Access-Control-Allow-Origin: "'*'"
ResponseTemplates:
application/json: ''
PassthroughBehavior: NEVER
RequestTemplates:
application/json: '{"statusCode": 200}'
Type: MOCK
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: true
method.response.header.Access-Control-Allow-Methods: true
method.response.header.Access-Control-Allow-Origin: true
DependsOn:
- ProxyResourceTagDELETE
- apiauth
ProxyResourceTagLinkGET:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref taglink
HttpMethod: GET
AuthorizationType: COGNITO_USER_POOLS
AuthorizerId: !Ref apiauth
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${LambdaARN}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceTagPOST
- apiauth
ProxyResourceTagLinkPOST:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref taglink
HttpMethod: POST
AuthorizationType: COGNITO_USER_POOLS
AuthorizerId: !Ref apiauth
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${LambdaARN}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceTagLinkGET
- apiauth
ProxyResourceTagLinkDELETE:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref taglink
HttpMethod: DELETE
AuthorizationType: COGNITO_USER_POOLS
AuthorizerId: !Ref apiauth
Integration:
Type: AWS_PROXY
IntegrationHttpMethod: POST
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${LambdaARN}/invocations
IntegrationResponses:
-
StatusCode: 200
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
DependsOn:
- ProxyResourceTagLinkPOST
- apiauth
OptionsMethodTagLink:
Type: AWS::ApiGateway::Method
Properties:
AuthorizationType: COGNITO_USER_POOLS
AuthorizerId: !Ref apiauth
RestApiId: !Ref ApplicationAPI
ResourceId: !Ref taglink
HttpMethod: OPTIONS
Integration:
IntegrationResponses:
-
StatusCode: 200
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
method.response.header.Access-Control-Allow-Methods: "'GET,POST,OPTIONS'"
method.response.header.Access-Control-Allow-Origin: "'*'"
ResponseTemplates:
application/json: ''
PassthroughBehavior: NEVER
RequestTemplates:
application/json: '{"statusCode": 200}'
Type: MOCK
MethodResponses:
-
StatusCode: 200
ResponseModels:
application/json: Empty
ResponseParameters:
method.response.header.Access-Control-Allow-Headers: true
method.response.header.Access-Control-Allow-Methods: true
method.response.header.Access-Control-Allow-Origin: true
DependsOn:
- ProxyResourceTagLinkDELETE
- apiauth
Deployment:
DependsOn: "OptionsMethodTagLink"
Type: AWS::ApiGateway::Deployment
Properties:
RestApiId:
Ref: "ApplicationAPI"
Description: "Application API deployment"
StageName: "prod"
Outputs:
APIBaseUrl:
Description: Application API Gateway base URL
Value: !Join
- ''
- - 'https://'
- !Ref ApplicationAPI
- '.execute-api.'
- !Ref AWS::Region
- '.amazonaws.com/prod'
APILinkUrl:
Description: Application API Gateway link endpoint URL
Value: !Join
- ''
- - 'https://'
- !Ref ApplicationAPI
- '.execute-api.'
- !Ref AWS::Region
- '.amazonaws.com/prod/link'
APITagUrl:
Description: Application API Gateway tag endpoint URL
Value: !Join
- ''
- - 'https://'
- !Ref ApplicationAPI
- '.execute-api.'
- !Ref AWS::Region
- '.amazonaws.com/prod/tag'
APITagLinkUrl:
Description: Application API Gateway taglink endpoint URL
Value: !Join
- ''
- - 'https://'
- !Ref ApplicationAPI
- '.execute-api.'
- !Ref AWS::Region
- '.amazonaws.com/prod/taglink'
Step 7: Upload the Child CloudFormation Templates to the S3 Bucket
Upload the child CFTs to the s3 bucket with the following commands:
$ aws s3 cp api.yml s3://$BUCKET $ aws s3 cp cognito.yml s3://$BUCKET $ aws s3 cp database.yml s3://$BUCKET $ aws s3 cp lambda.yml s3://$BUCKET
Step 8: Create the test.sh script
If you haven’t created the test.sh script (it should be the same as in Creating an API with AWS: Part 5: DELETE methods), create it with the following code:
BASEURL=$1
echo "=========================="
echo "Test POST /link link=https://www.allthecoding.com, tag=blog"
echo $HEADERS
resp=$(curl -X "POST" -H "Content-Type: application/json" -H "Authorization: $TOKEN" -d "{\"link\": \"https://www.allthecoding.com\", \"tag\": \"blog\"}" $BASEURL/link)
echo $resp
link_id1=$(echo $resp | jq -r '.link_id')
echo $link_id1
echo "=========================="
echo "Test GET /tag tag=blog"
resp=$(curl -X "GET" -H "Content-Type: application/json" -H "Authorization: $TOKEN" $BASEURL/tag?tag=blog)
echo $resp
tag_id1=$(echo $resp | jq -r '.[0].tag_id')
echo $tag_id1
echo "=========================="
echo "Test POST /tag tag=stuff"
resp=$(curl -X "POST" -H "Content-Type: application/json" -H "Authorization: $TOKEN" -d "{\"tag\": \"stuff\"}" $BASEURL/tag)
echo $resp
tag_id2=$(echo $resp | jq -r '.tag_id')
echo $tag_id2
echo "=========================="
echo "Test POST /link link=https://www.google.com, tag=stuff"
resp=$(curl -X "POST" -H "Content-Type: application/json" -H "Authorization: $TOKEN" -d "{\"link\": \"https://www.google.com\", \"tag\": \"stuff\"}" $BASEURL/link)
echo $resp
link_id2=$(echo $resp | jq -r '.link_id')
echo $link_id2
echo "=========================="
echo "Test POST /link link=https://www.google.com, tag_id=$tag_id1"
resp=$(curl -X "POST" -H "Content-Type: application/json" -H "Authorization: $TOKEN" -d "{\"link\": \"https://www.google.com\", \"tag_id\": \"$tag_id1\"}" $BASEURL/link)
echo $resp
link_id2=$(echo $resp | jq -r '.link_id')
echo $link_id2
echo "=========================="
echo "Test POST /link link=https://www.google.com, tag_id=invalid". Expect 400 response: Unknown tag_id invalid
resp=$(curl -X "POST" -H "Content-Type: application/json" -H "Authorization: $TOKEN" -d "{\"link\": \"https://www.google.com\", \"tag_id\": \"invalid\"}" $BASEURL/link)
echo $resp
echo "=========================="
echo "Test GET /link"
resp=$(curl -X "GET" -H "Content-Type: application/json" -H "Authorization: $TOKEN" $BASEURL/link)
echo $resp
echo "=========================="
echo "Test GET /link tag_id=$tag_id1"
resp=$(curl -X "GET" -H "Content-Type: application/json" -H "Authorization: $TOKEN" $BASEURL/link?tag_id=$tag_id1)
echo $resp
echo "=========================="
echo "Test GET /link tag=blog"
resp=$(curl -X "GET" -H "Content-Type: application/json" -H "Authorization: $TOKEN" $BASEURL/link?tag=blog)
echo $resp
echo "=========================="
echo "Test GET /link link_id=$link_id1"
resp=$(curl -X "GET" -H "Content-Type: application/json" -H "Authorization: $TOKEN" $BASEURL/link?link_id=$link_id1)
echo $resp
echo "=========================="
echo "Test GET /tag tag_id=$tag_id1"
resp=$(curl -X "GET" -H "Content-Type: application/json" -H "Authorization: $TOKEN" $BASEURL/tag?tag_id=$tag_id1)
echo $resp
echo "=========================="
echo "Test GET /tag"
resp=$(curl -X "GET" -H "Content-Type: application/json" -H "Authorization: $TOKEN" $BASEURL/tag)
echo $resp
echo "=========================="
echo "Test POST /taglink tag_id=$tag_id1, link_id=$link_id1"
resp=$(curl -X "POST" -H "Content-Type: application/json" -H "Authorization: $TOKEN" -d "{\"link_id\": \"$link_id1\", \"tag_id\": \"$tag_id1\"}" $BASEURL/taglink)
echo $resp
echo "=========================="
echo "Test POST /taglink tag_id=invalid, link_id=$link_id1. Expect 400 response: Unknown tag_id invalid"
resp=$(curl -X "POST" -H "Content-Type: application/json" -H "Authorization: $TOKEN" -d "{\"link_id\": \"$link_id1\", \"tag_id\": \"invalid\"}" $BASEURL/taglink)
echo $resp
echo "=========================="
echo "Test POST /taglink tag_id=$tag_id1, link_id=invalid. Expect 400 response: Unknown link_id invalid"
resp=$(curl -X "POST" -H "Content-Type: application/json" -H "Authorization: $TOKEN" -d "{\"link_id\": \"invalid\", \"tag_id\": \"$tag_id1\"}" $BASEURL/taglink)
echo $resp
echo "=========================="
echo "Test DELETE /link link_id=$link_id1"
resp=$(curl -X "DELETE" -H "Content-Type: application/json" -H "Authorization: $TOKEN" $BASEURL/link?link_id=$link_id1)
echo $resp
echo "=========================="
echo "Test DELETE /tag tag_id=$tag_id1"
resp=$(curl -X "DELETE" -H "Content-Type: application/json" -H "Authorization: $TOKEN" $BASEURL/tag?tag_id=$tag_id1)
echo $resp
echo "=========================="
echo "Test DELETE /taglink tag_id=$tag_id2, link_id=$link_id2"
resp=$(curl -X "DELETE" -H "Content-Type: application/json" -H "Authorization: $TOKEN" "$BASEURL/taglink?tag_id=$tag_id2&link_id=$link_id2")
echo $resp
Step 9: Create an authentication.sh script
If you haven’t created the authentication.sh script (it should be the same as in Creating an API with AWS: Part 5: DELETE methods), create it with the code below:
EMAIL=$1
PASSWORD=$2
aws cognito-idp admin-create-user --user-pool-id $POOL_ID --username $EMAIL
aws cognito-idp admin-set-user-password --user-pool-id $POOL_ID --username $EMAIL --password $PASSWORD --permanent
resp=$(aws cognito-idp describe-user-pool-client --client-id $CLIENT_ID --user-pool-id $POOL_ID)
CLIENT_SECRET=$(echo $resp | jq -r '.UserPoolClient.ClientSecret')
SECRET_HASH=$(python get_client_secret.py $EMAIL $CLIENT_ID $CLIENT_SECRET | awk -F: '{print $2}' | xargs)
resp=$(aws cognito-idp initiate-auth --region us-east-1 --auth-flow USER_PASSWORD_AUTH --client-id $CLIENT_ID --auth-parameters USERNAME=$EMAIL,PASSWORD=$PASSWORD,SECRET_HASH=$SECRET_HASH)
TOKEN=$(echo $resp | jq -r '.AuthenticationResult.IdToken')
echo $TOKEN
export TOKEN=$TOKEN
Create a python script called get_client_secret.py (if you haven’t already) and add the following code:
import sys
import hmac, hashlib, base64
username = sys.argv[1]
app_client_id = sys.argv[2]
key = sys.argv[3]
message = bytes(sys.argv[1]+sys.argv[2], 'utf-8')
key = bytes(sys.argv[3], 'utf-8')
secret_hash = base64.b64encode(hmac.new(key, message, digestmod=hashlib.sha256).digest()).decode()
print("SECRET HASH:", secret_hash)
Step 10: Launch the CloudFormation Template
Launch the nested CFT with this command:
$ aws cloudformation create-stack --stack-name nested --template-body file://nested.yml --parameters ParameterKey=BucketName,ParameterValue=$BUCKET --capabilities CAPABILITY_NAMED_IAM
Poll for its progress with:
$ aws cloudformation describe-stacks --stack-name nested
When complete, make a note of the APIBaseUrl, UserPoolID and APIClientID values. Export them to environment variables with these commands. Be sure the replace the values in angle brackets with the real values:
$ export BASEURL=<APIBaseUrl> $ export POOL_ID=<UserPoolID> $ export CLIENT_ID=<APIClientID>
Step 11: Run the authentication.sh script
Run the authentication.sh script (from Step 9) to generate an auth token and export it to the environment. Replace ‘EMAIL’ and ‘PASSWORD’ with an email address for the user and and a secure password:
$ source ./authentication.sh EMAIL PASSWORD
Step 12: Run the test.sh script
Run the test.sh script (from Step 8) to check that the API is working correctly:
$ ./test.sh $BASEURL
If all goes well, the script should run with no errors other than those expected (as detailed in the output).
Step 9: Clean Up
If you don’t want to keep your API (note that AWS will charge you for it if you don’t have Free Tier on your account), delete the CFT stack (all resources created) by running:
$ aws cloudformation delete-stack --stack-name nested
Empty and delete your S3 bucket. To do this, run:
$ aws s3 rm s3://$BUCKET/ --recursive $ aws s3api delete-bucket --bucket $BUCKET
You will then need to remove any logs that were generated by the Lambda function. To do this:
- Log into the AWS Console (your AWS account) using your Administrator IAM user, and navigate to CloudWatch.
- On the left-hand side, select Logs -> Log Groups.
- Select ‘aws/lambda/requestHandler’.
- Under Actions, select ‘Delete log groups’ and confirm.
Conclusion
You should now be able to create a simple API on AWS with a nested CloudFormation template, with a separate CloudFormation template for each group of resources.
Thanks for reading!
1 Response
[…] Creating an API with AWS: Part 6: Nested CloudFormation Templates January 28, 2022 […]