Skip to content

Latest commit

 

History

History
159 lines (111 loc) · 5.28 KB

README.md

File metadata and controls

159 lines (111 loc) · 5.28 KB

RAG Application CDK Python project!

rag_with_opensearch_arch

This is an QA application with LLMs and RAG project for CDK development with Python.

The cdk.json file tells the CDK Toolkit how to execute your app.

This project is set up like a standard Python project. The initialization process also creates a virtualenv within this project, stored under the .venv directory. To create the virtualenv it assumes that there is a python3 (or python for Windows) executable in your path with access to the venv package. If for any reason the automatic creation of the virtualenv fails, you can create the virtualenv manually.

To manually create a virtualenv on MacOS and Linux:

$ python3 -m venv .venv

After the init process completes and the virtualenv is created, you can use the following step to activate your virtualenv.

$ source .venv/bin/activate

If you are a Windows platform, you would activate the virtualenv like this:

% .venv\Scripts\activate.bat

Once the virtualenv is activated, you can install the required dependencies.

(.venv) $ pip install -r requirements.txt

To add additional dependencies, for example other CDK libraries, just add them to your setup.py file and rerun the pip install -r requirements.txt command.

Before synthesizing the CloudFormation, you should set approperly the cdk context configuration file, cdk.context.json.

For example:

{
  "opensearch_domain_name": "open-search-domain-name",
  "jumpstart_model_info": {
    "model_id": "meta-textgeneration-llama-2-7b-f",
    "version": "2.0.1"
  }
}

ℹ️ The model_id, and version provided by SageMaker JumpStart can be found in SageMaker Built-in Algorithms with pre-trained Model Table.

⚠️ Important: Make sure you need to make sure docker daemon is running.
Otherwise you will encounter the following errors:

ERROR: Cannot connect to the Docker daemon at unix://$HOME/.docker/run/docker.sock. Is the docker daemon running?
jsii.errors.JavaScriptError:
  Error: docker exited with status 1

Now this point you can now synthesize the CloudFormation template for this code.

(.venv) $ export CDK_DEFAULT_ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
(.venv) $ export CDK_DEFAULT_REGION=$(aws configure get region)
(.venv) $ cdk synth --all

Now we will be able to deploy all the CDK stacks at once like this:

(.venv) $ cdk deploy --require-approval never --all

Or, we can provision each CDK stack one at a time like this:

Step 1: List all CDK Stacks

(.venv) $ cdk list
RAGVpcStack
RAGOpenSearchStack
RAGSageMakerStudioStack
EmbeddingEndpointStack
LLMEndpointStack
StreamlitAppStack

Step 2: Create OpenSearch cluster

(.venv) $ cdk deploy --require-approval never RAGVpcStack RAGOpenSearchStack

Step 3: Create SageMaker Studio

(.venv) $ cdk deploy --require-approval never RAGSageMakerStudioStack

Step 4: Deploy LLM Embedding Endpoint

(.venv) $ cdk deploy --require-approval never EmbeddingEndpointStack

Step 5: Deploy Text Generation LLM Endpoint

(.venv) $ cdk deploy --require-approval never LLMEndpointStack

Step 6 (Optional): Deploy the Streamlit app on ECS Fargate

⚠️ Before deploy the following CDK stack, make sure Docker is runing on your machine.

(.venv) $ cdk deploy --require-approval never StreamlitAppStack

Once all CDK stacks have been successfully created, proceed with the remaining steps of the overall workflow.

Clean Up

Delete the CloudFormation stacks by running the below command.

(.venv) $ cdk destroy --all

Useful commands

  • cdk ls list all stacks in the app
  • cdk synth emits the synthesized CloudFormation template
  • cdk deploy deploy this stack to your default AWS account/region
  • cdk diff compare deployed stack with current state
  • cdk docs open CDK documentation

Enjoy!

References