AWS-Redshift-Serverless-with-AWS-SDK-JS-V3

Deployment and Modeling of AWS Redshift Serverless (A cloud data warehouse).

1. The NodeJS script (index.js) creates and deletes the following resources:

   • Redshift Serverless Namespace
   • Redshift Serverless Workgroup
   • Redshift Serverless Usage Limit

2. Additional SQL scripts in the repo for modeling include:

   • DDL script (ddl.sql) for data objects (TABLES) creation. Run via Redshift Query Editor 2
   • DML script (dml-v1.sql) for inserting data. Run via either:
     • Redshift Query Editor 2
     • NodeJS module - @aws-sdk/client-redshift-data (AWS-SDK-JS-V3)
       
   • DQL script (dql.sql). Run via Redshift Query Editor 2

DEPLOYING AWS Redshift Serverless with the NodeJS script

Option 1: Clone to Local Computer

To deploy the stack on AWS, follow these steps:

1. Install NodeJS and relevant NodeJS AWS-SDK V3 modules, assuming Ubuntu or Debian Linux OS

   • curl -fsSL https://deb.nodesource.com/setup_19.x | sudo -E bash -
     
   • sudo apt-get install -y nodejs
     
   • sudo npm install @aws-sdk/client-redshift-serverless
   • sudo npm install @aws-sdk/client-redshift-data

2. Download or clone the following files, from this repo, into the current working directory (CWD):
   

   • NodeJS script: index.js
     
   • JSON files: credentials.json and inputConfig.json
     

3. Fill in relevant values in the inputConfig.json file.
   

   • References for inputConfig.json :
     • https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-redshift-serverless/classes/createnamespacecommand.html
     • https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-redshift-serverless/classes/createworkgroupcommand.html
     • https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-redshift-serverless/classes/createusagelimitcommand.html
     • https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-redshift-serverless/classes/deleteworkgroupcommand.html

4. Then run the code, assuming sudo access:
   

   • sudo node index.js
     
     

OPTION 2: Through GitHub Actions

To deploy on AWS, follow these steps:

1. Check and fill relevant values in the GitHub Actions YML deployment file.

   • Link: https://github.com/MongoExpUser/AWS-Redshift-Serverless-with-AWS-SDK-JS-V3/blob/main/.github/workflows/deploy-redshift-sless.yml

2. Also fill relevant values in the inputConfig.json file.

   • Ensure that the environment (dev, stag or prod) and region in the file (inputConfig.json) correspond to the values in the GitHub Actions YML file.
     
   • Add the actual values of the AWS IAM Role(s), AWS VPC Security Group id(s) and AWS VPC subnet ids to the GitHub Secrets instead of specifying them in the config file (inputConfig.json) to avoid exposing their values.
   • See the env sections of the YML Deployment File: https://github.com/MongoExpUser/AWS-Redshift-Serverless-with-AWS-SDK-JS-V3/blob/main/.github/workflows/deploy-redshift-sless.yml

3. Add the actual values for credentials to the GitHub Secrets.

   • These include: accessKeyId, secretAccessKey and region.
   • This prevents exposure of the credentials.

4. Then enable GitHub Actions Workflow and run the YML file.

   • Link: https://github.com/MongoExpUser/AWS-Redshift-Serverless-with-AWS-SDK-JS-V3/actions
     

License

Copyright © 2023. MongoExpUser

Licensed under the MIT license.