Skip to content

Commit 7d19e11

Browse files
ajosh0504ajosh0504
authored
Merge pull request #75 from fmenezes/localai
Add local ai bot
2 parents f1f7a2e + 28ad1d9 commit 7d19e11

File tree

5 files changed

+421
-0
lines changed

5 files changed

+421
-0
lines changed

‎apps/local-bot/Dockerfile

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
FROM python:3.12
2+
WORKDIR /opt/app
3+
ADD requirements.txt .
4+
RUN pip install -r requirements.txt
5+
ADD app.py .
6+
7+
CMD ["streamlit", "run", "app.py", "--server.port=8501", "--server.address=0.0.0.0"]

‎apps/local-bot/README.md

+279
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,279 @@
1+
# Step-by-Step Guide: Building a Local Chatbot with Streamlit, LangChain, Ollama, and MongoDB Atlas
2+
3+
In this tutorial, we'll set up a local chatbot using **Streamlit**, **LangChain**, **Ollama**, and **MongoDB Atlas Search**. This bot will leverage MongoDB's powerful Atlas Search capabilities alongside local LLMs (Large Language Models) via Ollama, allowing you to enhance user queries with context from chat history.
4+
5+
## Prerequisites
6+
Before starting, make sure you have the following installed:
7+
8+
* Docker
9+
* Docker Compose
10+
11+
> [!NOTE]
12+
> For this tutorial, Docker is essential for containerized, isolated development.
13+
14+
## Step 1: Setting Up the Project
15+
16+
### Project Overview
17+
We’ll start by creating a directory for the project files and setting up our working structure.
18+
19+
```sh
20+
mkdir localai
21+
cd localai
22+
```
23+
24+
### Project Structure
25+
26+
Organize your project files as shown:
27+
28+
```
29+
localai/
30+
├── app.py
31+
├── Dockerfile
32+
├── compose.yaml
33+
└── requirements.txt
34+
```
35+
36+
### Tool Overview
37+
38+
Here’s a quick rundown of the tools we’re using in this project:
39+
40+
* *[Streamlit](https://streamlit.io)*: A Python library for easily creating data-based web applications. We'll use it to create a local chatbot interface.
41+
* *[LangChain](https://langchain.com)*: A framework that simplifies working with LLMs and document processing. It will assist processing user queries and generate responses.
42+
* *[Ollama](https://ollama.com)*: A solution for deploying LLMs locally without external API dependency. It to host our models.
43+
* *[MongoDB Atlas Search](https://www.mongodb.com/products/platform/atlas-search)*: Adds a powerful, flexible vector search functionality to our app. It will store user queries and responses in MongoDB.
44+
45+
### Setting Up `requirements.txt`
46+
47+
In `requirements.txt`, specify the dependencies needed for this project:
48+
49+
```requirements.txt
50+
streamlit
51+
ollama
52+
langchain
53+
langchain_ollama
54+
pymongo
55+
langchain_mongodb
56+
langchain_community
57+
markdownify
58+
```
59+
60+
### Docker Configuration
61+
62+
#### `Dockerfile`
63+
Create a Dockerfile and add the following content. This file will define the container setup, ensuring our app and its dependencies run consistently across environments.
64+
65+
```Dockerfile
66+
FROM python:3.12
67+
WORKDIR /opt/app
68+
ADD requirements.txt .
69+
RUN pip install -r requirements.txt
70+
ADD app.py .
71+
72+
CMD ["streamlit", "run", "app.py", "--server.port=8501", "--server.address=0.0.0.0"]
73+
```
74+
75+
#### `compose.yaml`
76+
Define your Docker Compose configuration in `compose.yaml`:
77+
78+
```yaml
79+
services:
80+
app:
81+
build:
82+
context: .
83+
ports:
84+
- 8501:8501/tcp
85+
environment:
86+
MONGO_URI: mongodb://root:root@mongo:27017/admin?directConnection=true
87+
ollama:
88+
image: ollama/ollama
89+
mongo:
90+
image: mongodb/mongodb-atlas-local
91+
environment:
92+
- MONGODB_INITDB_ROOT_USERNAME=root
93+
- MONGODB_INITDB_ROOT_PASSWORD=root
94+
ports:
95+
- 27017:27017
96+
```
97+
98+
> [!TIP]
99+
> If you are on macOS, install Ollama locally and use this modified version of `compose.yaml`:
100+
> ```yaml
101+
> services:
102+
> app:
103+
> build:
104+
> context: .
105+
> ports:
106+
> - 8501:8501/tcp
107+
> environment:
108+
> OLLAMA_HOST: host.docker.internal:11434
109+
> MONGO_URI: mongodb://root:root@mongo:27017/admin?directConnection=true
110+
> extra_hosts:
111+
> - "host.docker.internal:host-gateway"
112+
> mongo:
113+
> image: mongodb/mongodb-atlas-local
114+
> environment:
115+
> - MONGODB_INITDB_ROOT_USERNAME=root
116+
> - MONGODB_INITDB_ROOT_PASSWORD=root
117+
> ports:
118+
> - 27017:27017
119+
> ```
120+
121+
## Step 2: Creating the Initial App
122+
123+
Create app.py with a simple “Hello World” message to ensure your environment is set up correctly.
124+
125+
```python
126+
import streamlit as st
127+
128+
st.write('Hello, World!')
129+
```
130+
131+
With all files in place, you can now build and run the Docker containers:
132+
133+
```sh
134+
docker compose up --build
135+
```
136+
137+
Access the app at http://localhost:8501 in your browser. You should see the message Hello, World!
138+
139+
> _Expected Output_: A browser page displaying “Hello, World!” confirms your setup is correct.
140+
141+
## Step 3: Build the Chatbot
142+
143+
### Step 3.1: Setting Up MongoDB and Ollama
144+
145+
In `app.py`, connect to MongoDB and Ollama. This will pull LLM models from Ollama and set up MongoDB for data storage.
146+
147+
```python
148+
import os
149+
import streamlit as st
150+
from pymongo import MongoClient
151+
import ollama
152+
153+
# Model and embedding configurations
154+
MODEL = "llama3.2"
155+
EMBEDDING_MODEL = "nomic-embed-text"
156+
MONGO_URI = os.getenv("MONGO_URI", "mongodb://localhost:27017")
157+
158+
# Pull models from Ollama
159+
ollama.pull(MODEL)
160+
ollama.pull(EMBEDDING_MODEL)
161+
162+
# Initialize MongoDB client
163+
try:
164+
mongo_client = MongoClient(MONGO_URI)
165+
collection = mongo_client["bot"]["data"]
166+
except Exception as e:
167+
st.error(f"Failed to connect to MongoDB: {e}")
168+
st.stop()
169+
```
170+
171+
> [!NOTE]
172+
> After each step, you can test the app by re-running `docker compose up --build`.
173+
174+
### Step 3.2: Loading Documents and Creating a Vector Search Index
175+
176+
Now, load documents, process them with LangChain, and store them as vector embeddings in MongoDB. This setup allows MongoDB Atlas to perform fast vector-based searches.
177+
178+
```python
179+
from langchain_ollama import OllamaEmbeddings
180+
from langchain_community.document_loaders import WebBaseLoader
181+
from langchain_community.document_transformers import MarkdownifyTransformer
182+
from langchain_text_splitters import RecursiveCharacterTextSplitter
183+
from langchain_mongodb import MongoDBAtlasVectorSearch
184+
185+
embedding = OllamaEmbeddings(model=EMBEDDING_MODEL)
186+
collection.drop()
187+
188+
loaders = [
189+
WebBaseLoader("https://en.wikipedia.org/wiki/AT%26T"),
190+
WebBaseLoader("https://en.wikipedia.org/wiki/Bank_of_America")
191+
]
192+
docs = []
193+
for loader in loaders:
194+
for doc in loader.load():
195+
docs.append(doc)
196+
197+
md = MarkdownifyTransformer()
198+
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
199+
converted_docs = md.transform_documents(docs)
200+
splits = text_splitter.split_documents(converted_docs)
201+
202+
vectorstore = MongoDBAtlasVectorSearch.from_documents(splits, embedding, collection=collection, index_name="default")
203+
vectorstore.create_vector_search_index(768)
204+
```
205+
206+
> _Expected Output_: After this step, MongoDB Atlas should contain indexed documents, enabling fast vector-based search capabilities.
207+
208+
### Step 3.3: Setting Up the Chat Model
209+
210+
Next, set up the chat model with a retrieval mechanism and define the chain of operations that will handle user queries.
211+
212+
```python
213+
from langchain_ollama import ChatOllama
214+
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
215+
from langchain_core.output_parsers import StrOutputParser
216+
from langchain_core.runnables.history import RunnableWithMessageHistory
217+
from langchain_core.chat_history import BaseChatMessageHistory
218+
from langchain_mongodb import MongoDBChatMessageHistory
219+
220+
retriever = vectorstore.as_retriever()
221+
chat = ChatOllama(model=MODEL)
222+
223+
# System message for the chatbot
224+
SYSTEM_MESSAGE = """You're a helpful assistant. Answer all questions to the best of your ability. If you don't know the answer let the user know to find help on the internet.
225+
226+
Available context:
227+
{context}
228+
"""
229+
230+
prompt_template = ChatPromptTemplate.from_messages([
231+
("system", SYSTEM_MESSAGE),
232+
MessagesPlaceholder("history"),
233+
("human", "{input}"),
234+
])
235+
236+
chain = {
237+
"context": itemgetter("input") | retriever,
238+
"input": itemgetter("input"),
239+
"history": itemgetter("history")
240+
} | prompt_template | chat | StrOutputParser()
241+
242+
def get_session_history(session_id: str) -> BaseChatMessageHistory:
243+
return MongoDBChatMessageHistory(MONGO_URI, session_id, database_name="bot")
244+
245+
history_chain = RunnableWithMessageHistory(chain, get_session_history, input_messages_key="input", history_messages_key="history")
246+
```
247+
248+
### Step 3.4: Creating the Chat Interface
249+
250+
Now, use Streamlit to create a chat interface for interacting with the chatbot.
251+
252+
```python
253+
st.title("Chatbot")
254+
st.caption("A Streamlit chatbot")
255+
256+
history = get_session_history()
257+
for msg in history.messages:
258+
st.chat_message(msg.type).write(msg.content)
259+
260+
if prompt := st.chat_input():
261+
st.chat_message("user").write(prompt)
262+
with st.chat_message("ai"):
263+
with st.spinner("Thinking..."):
264+
st.write_stream(history_chain.stream({"input": prompt}))
265+
```
266+
267+
At this point, you can start prompting with inputs like “Who started AT&T?” and see the chatbot respond!
268+
269+
## Conclusion and Next Steps
270+
271+
In this tutorial, we built a local chatbot setup using MongoDB Atlas Search and local LLMs via Ollama, integrated through Streamlit. This project forms a robust foundation for further development and deployment.
272+
273+
Possible Extensions:
274+
275+
* Add more sophisticated pre-processing for documents.
276+
* Experiment with different models in Ollama.
277+
* Deploy to a cloud environment like AWS or Azure for production scaling.
278+
279+
Feel free to customize this setup to suit your needs. Happy coding!

‎apps/local-bot/app.py

+106
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
import os
2+
from operator import itemgetter
3+
4+
import streamlit as st
5+
import ollama
6+
from langchain_core.output_parsers import StrOutputParser
7+
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
8+
from langchain_ollama import ChatOllama, OllamaEmbeddings
9+
from pymongo import MongoClient
10+
from langchain_mongodb import MongoDBChatMessageHistory, MongoDBAtlasVectorSearch
11+
from langchain_community.document_loaders import WebBaseLoader
12+
from langchain_community.document_transformers import MarkdownifyTransformer
13+
from langchain_text_splitters import RecursiveCharacterTextSplitter
14+
from langchain_core.runnables.history import RunnableWithMessageHistory
15+
from langchain_core.chat_history import BaseChatMessageHistory
16+
17+
# System message for the chatbot
18+
SYSTEM_MESSAGE = """You're a helpful assistant. Answer all questions to the best of your ability. If you don't know the answer let the user know to find help on the internet.
19+
20+
Available context:
21+
{context}
22+
"""
23+
24+
# Model and embedding configurations
25+
MODEL = "llama3.2"
26+
EMBEDDING_MODEL = "nomic-embed-text"
27+
MONGO_URI = os.getenv("MONGO_URI", "mongodb://localhost:27017")
28+
29+
# Pull models from Ollama
30+
ollama.pull(MODEL)
31+
ollama.pull(EMBEDDING_MODEL)
32+
33+
# Initialize MongoDB client
34+
try:
35+
mongo_client = MongoClient(MONGO_URI)
36+
collection = mongo_client["bot"]["data"]
37+
except Exception as e:
38+
st.error(f"Failed to connect to MongoDB: {e}")
39+
st.stop()
40+
41+
# Initialize embeddings
42+
embedding = OllamaEmbeddings(model=EMBEDDING_MODEL)
43+
44+
# Load documents and create vector search index if not already present
45+
collection.drop()
46+
47+
loaders = [
48+
WebBaseLoader("https://en.wikipedia.org/wiki/AT%26T"),
49+
WebBaseLoader("https://en.wikipedia.org/wiki/Bank_of_America")
50+
]
51+
docs = []
52+
for loader in loaders:
53+
for doc in loader.load():
54+
docs.append(doc)
55+
md = MarkdownifyTransformer()
56+
text_splitter = RecursiveCharacterTextSplitter(
57+
chunk_size=1000, chunk_overlap=200)
58+
docs = loader.load()
59+
converted_docs = md.transform_documents(docs)
60+
splits = text_splitter.split_documents(converted_docs)
61+
vectorstore = MongoDBAtlasVectorSearch.from_documents(
62+
splits, embedding, collection=collection, index_name="default")
63+
vectorstore.create_vector_search_index(768)
64+
65+
# Initialize retriever and chat model
66+
retriever = vectorstore.as_retriever()
67+
chat = ChatOllama(model=MODEL)
68+
69+
# Define prompt template
70+
prompt_template = ChatPromptTemplate.from_messages([
71+
("system", SYSTEM_MESSAGE),
72+
MessagesPlaceholder("history"),
73+
("human", "{input}"),
74+
])
75+
76+
# Define the chain of operations
77+
chain = {
78+
"context": itemgetter("input") | retriever,
79+
"input": itemgetter("input"),
80+
"history": itemgetter("history")
81+
} | prompt_template | chat | StrOutputParser()
82+
83+
# Function to get session history
84+
def get_session_history() -> BaseChatMessageHistory:
85+
return MongoDBChatMessageHistory(MONGO_URI, "user", database_name="bot")
86+
87+
88+
# Initialize history chain
89+
history_chain = RunnableWithMessageHistory(
90+
chain, get_session_history, input_messages_key="input", history_messages_key="history")
91+
92+
# Streamlit UI
93+
st.title("Chatbot")
94+
st.caption("A Streamlit chatbot")
95+
96+
# Display chat history
97+
history = get_session_history()
98+
for msg in history.messages:
99+
st.chat_message(msg.type).write(msg.content)
100+
101+
# Handle user input
102+
if prompt := st.chat_input():
103+
st.chat_message("user").write(prompt)
104+
with st.chat_message("ai"):
105+
with st.spinner("Thinking..."):
106+
st.write_stream(history_chain.stream({"input": prompt}))

0 commit comments

Comments
 (0)