S3 Storage Lambda

OverOps can use AWS Lambda for a serverless Variable Store on S3. A CloudFormation stack is used to facilitate the deployment of these services.

Prerequisites

git - in order to clone the repository with the CloudFormation template
docker - we use docker to encapsulate the dependencies for deploying to AWS
S3 bucket - an S3 bucket for the overops storage
IAM Credentials - AWS credentials, which we'll copy into the deploy folder. If you already have credentials in ~/.aws/credentials file you may use this file.

The credentials must have appropriate access to AWS services to read/write S3, API gateway, lambda and create/manage roles to apply to those as well as CloudFormation itself.
The required roles are as follows:

AWSLambdaFullAccess
IAMFullAccess
AmazonS3FullAccess
AmazonAPIGatewayAdministrator
AWSCloudFormationFullAccess

These may be applied in AWS IAM (Identity and Access Management):
https://console.aws.amazon.com/iam/home

Setup / Configuration

The first thing you will need to do is clone the takipi-storage repository from github and checkout the branch with the lambda implementation.

git clone https://github.com/takipi/takipi-storage
git checkout s3-storage-lambda

After completing the above, the AWS credentials need to be copied into takipi-storage/deploy/docker/credentials.

Enter the name of the S3 bucket to be used for storage in takipi-storage/deploy/docker/Dockerfile:

ENV S3_BUCKET=your-overops-storage

If you are not deploying to us-east-1, you will need to edit takipi-storage/deploy/docker/config:

region=us-east-1

Deploy

The deploy script (deploy.sh) will package and deploy the CloudFormation stack for the overops-storage S3 Lambda services.

To execute the script make sure you are in the deploy directory before running it from a terminal.
For example:

cd takipi-storage/deploy
./deploy.sh

When the script completes successfully, you should see output in the console with the storage URL, similar to what is seen below.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------
Outputs                                                                                                                                                         
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
Key                 StorageApi                                                                                                                                  
Description         URL for application                                                                                                                         
Value               https://we7lby9gxj.execute-api.us-east-1.amazonaws.com/Prod                                                                                 
-----------------------------------------------------------------------------------------------------------------------------------------------------------------

To view and manage what is deployed directly on AWS, navigate to https://console.aws.amazon.com/cloudformation/home.

Before finishing the setup, verify the deployment is healthy by using the StorageApi URL from the output above and appending "/storage/v1/diag/ping" to it.

For example: https://we7lby9gxj.execute-api.us-east-1.amazonaws.com/Prod/storage/v1/diag/ping

The response should give an "ok" indicating that the services are deployed and operational.

Using the Storage Server

Use the StorageApi URL (eg https://we7lby9gxj.execute-api.us-east-1.amazonaws.com/Prod) for the "Host name" in the OverOps Management Console.

Once entered, test the connection and then save it. Your OverOps installation is now configured to use AWS Lambda for handling all storage requests with S3.