Deploying the api for serverless

How to deploy the apydox api to AWS Lambda and API Gateway

Prerequisites

You will need the following tools and accounts in place in order to deploy the apydox portal as serverless functions in AWS.

• An AWS account - https://aws.amazon.com/

• AWS CLI - https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html

• AWS SAM CLI - https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html

• Ensure you configure the AWS region correctly on your machine. For macOS and linux you could use ~/.bash_profile or ~/.profile to export the correct region. e.g `export AWS_REGION=eu-west-1`

• jq >=1.5 - A CLI tool used by the deployment script to process JSON https://stedolan.github.io/jq/

• Building from source (Currently the only way to build)

  • Make sure you have Go installed on your machine. https://golang.org

  • Clone the apydox project repo and checkout to the version you want to deploy. https://github.com/freshwebio/apydox

Creating the secret store

In AWS Console

Firstly login to the console by heading to https://console.aws.amazon.com/.

Once you have logged in you will be taken to a dashboard where you can search for the secrets manager service like the following screenshot shows.

Once you have loaded the secrets manager UI, click the button that will allow you to store a new secret. This should take you to a page that looks something like the following:

Now select the "Other type of secrets" option and enter the secret keys and their values. The two secrets you will need for the apydox api are the following:

{
  "APYDOX_API_GITHUB_CLIENT_ID": "your_client_id",
  "APYDOX_API_GITHUB_CLIENT_SECRET": "your_client_secret"
}

You can see the format above by clicking the Plaintext tab, this will reveal a JSON object for all the key value pairs.

You will need to get these credentials from the Github OAuth application you create for your instance of apydox.

Click next and enter a name such as FRESHWEB/APYDOX-API-SECRETS and click next to review and save your new secret store.

Once you have saved your secret you will want to take note of the ARN, it will be something like the following:

arn:aws:secretsmanager:{aws_region}:{account_number}:secret:FRESHWEB/APYDOX-API-SECRETS-FaljYa

With the AWS CLI

Create a file that holds the JSON object for the secrets in the apydox/api directory like the following:

secrets-deploy.json

{
  "APYDOX_API_GITHUB_CLIENT_ID": "your_client_id",
  "APYDOX_API_GITHUB_CLIENT_SECRET": "your_client_secret"
}

Now that you have your file created with the secrets needed to connect to Github, you can go ahead and run the following command (Replace the name and the file path that contains the secrets as you see fit):

aws secretsmanager create-secret --name MYCOMPANY/APYDOX-API-SECRETS --secret-string file://secrets-deploy.json

This will then output something like the following:

{
    "SecretARN": "arn:aws:secretsmanager:region:accountid:secret:MYCOMPANY/APYDOX-API-SECRETS-AbCdEf",
    "SecretName": "MYCOMPANY/APYDOX-API-SECRETS",
    "SecretVersionId": "EXAMPLE1-90ab-cdef-fedc-ba987EXAMPLE"
}

Keep a note of the value in the SecretARN field for when we set up the parameters

Setting up your parameters

There is an example file in the root directory of the project called sam-deploy-params.example.json, the first thing you will need to do is copy that file to sam-deploy-params.json. This file contains two example parameter values:

{
  "EnvType": "production",
  "SecretResourceARN": "arn:aws:secretsmanager:region:accountid:secret:MYCOMPANY/APYDOX-API-SECRETS-AbCdEf"
}

The ARN you took a note of in creating your secret store will need to be specified as the value for the SecretResourceARN key.

There are extra parameters that you can provide of which some are optional depending on your setup. These extra parameters are the following:

• SecretID (The default value is APYDOX/API-SECRETS)

  • The name of the secret to be injected as an environment variable into the lambda functions, if you named your secret store something other than the default value you will need to provide this parameter.

• SubnetID - https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html

• SecurityGroupID - https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html

There are other parameters defined in the template you don't need to worry about for deployments. These other parameters are only useful for running the api locally for the purpose of development.

Deploying the api

First of all the deployment process relies and some environment variables you must set, one is the name of the CloudFormation stack you will be creating and the second is the s3 bucket the packaged lambda functions should be saved to.

You will have to go into AWS console and create the S3 bucket to publish the packaged version of the serverless functions before deploying the api.

Once you have done this you can then ensure the environment variables are set by carrying out the following in your terminal:

export STACK_NAME=my-company-apydox-api
export AWS_S3_BUCKET=my-company-apydox-packaged

From within the api directory of the apydox repo run the following command:

make -f Makefile.lambda deploy

Wait a few minutes for AWS to report that all the resources have been created/updated in the stack.

This may fail with errors such as access denied for saving objects to S3 if the AWS credentials configured on your machine are for a different account or have expired.

Go into your AWS console, generate new keys and configure your machine with those credentials using "aws configure".

In the case the deployment is successful you can go into API Gateway via the AWS console and check everything is correct for the newly created API and map your custom domain or subdomain to your instance of the apydox api.

Setting up custom domain names for REST APIs - Amazon API GatewayAmazon API Gateway

Testing the api

To test the api is up and running, run a test with curl such as:

curl {your-api-url}/auth/github/check 

Alternatively you can carry out a GET request against {your-api-url}/auth/github/check in a tool like postman.

You will know the API is up and running if you receive a 401 Unauthorized response header with the following response body:

{
  "validToken": false
}

In the case you get a 500 server error instead, navigate to the CloudWatch logs for the lambda function with the name ApyDox-CheckAccessTokenFunction to see if it is a problem with access permissions.

Your SecretResourceARN may be incorrect as an example. It might seem correct on the surface but one thing you have to look out for is the need to ensure you create your secret store and deploy your serverless api in the same region.

Updating your serverless instance

To update your serverless instance, simply run through the steps again and make sure all of your secrets and parameters are set correctly. Once everything is in place, make sure you check out the repo to the new version you are upgrading to.