Skip to content

Create USER_GUIDE.md #55

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Create USER_GUIDE.md #55

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions USER_GUIDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# User Guide: OpenSearch Remote Vector Index Builder

## Getting Started

### 1. Provision an Instance
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should mention that an instance should have an IAM role attached, and link to the EC2 documentation that explains how to do that. And that IAM role must have permissions to read/write to the remote s3 bucket that gets attached with OpenSearch cluster

System requirements:
- NVIDIA GPU powered machine with CUDA Toolkit and Docker installed.
- ex. AWS EC2 `g5.2xlarge` instance running Deep Learning OSS Nvidia Driver AMI.

### 2. Clone the Repository
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If users are just running the Remote Vector Index Builder image, no need to clone the repo or build all of the images in sequence. All they need to do is do docker pull opensearchstaging/remote-vector-index-builder:api-latest. The reason i included those other steps in the developer guide was if someone wanted to make some modifications to the images, and test locally. That may need to be clarified there.

```bash
git clone https://github.com/opensearch-project/remote-vector-index-builder.git
cd remote-vector-index-builder
```

## Building Docker Images

There are 3 images that need to be built:

### 1. Build Faiss Base Image
First, initialize the Faiss submodule:
```bash
git submodule update --init
```

Then build the Faiss base image:
```bash
docker build -f ./base_image/build_scripts/Dockerfile . -t opensearchstaging/remote-vector-index-builder:faiss-base-latest
```

### 2. Build Core Image
```bash
docker build -f ./remote_vector_index_builder/core/Dockerfile . -t opensearchstaging/remote-vector-index-builder:core-latest
```

### 3. Build API Image
```bash
docker build -f ./remote_vector_index_builder/app/Dockerfile . -t opensearchstaging/remote-vector-index-builder:api-latest
```

## Running the Service

### Starting the Docker Container
```bash
docker run --gpus all -p 80:1025 opensearchstaging/remote-vector-index-builder:api-latest
```

The service will be available at the instance's public IP address on port 80.
For information on configuring OpenSearch to use this service, please refer to the [OpenSearch k-NN documentation.](https://docs.opensearch.org/docs/latest/vector-search/)