exercises/06-RAG.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Implement the Retrieval for a Retrieval Augmented Generation (RAG) Use Case\n",
    "\n",
    "Now that you have all your context information stored in the SAP HANA Cloud Vector Store, you can start asking the LLM questions about the orchestration service of Generative AI Hub. This time, the model will not respond from its knowledge base—what it learned during training—but instead, the retriever will search for relevant context information in your vector database and send the appropriate text chunk to the LLM to review before responding.\n",
    "\n",
    "👉 Change the `LLM_DEPLOYMENT_ID` in [variables.py](variables.py) to your deployment ID from exercise [01-explore-genai-hub](01-explore-genai-hub.md). For that go to **SAP AI Launchpad** application and navigate to **ML Operations** > **Deployments**.\n",
    "\n",
    "☝️ The `LLM_DEPLOYMENT_ID` is the deployment ID of the chat model you want to use e.g. **gpt-4o-mini**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import init_env\n",
    "import variables\n",
    "\n",
    "init_env.set_environment_variables()\n",
    "\n",
    "from gen_ai_hub.proxy.langchain.openai import ChatOpenAI\n",
    "from gen_ai_hub.proxy.langchain.openai import OpenAIEmbeddings\n",
    "\n",
    "from langchain.chains import RetrievalQA\n",
    "\n",
    "from langchain_community.vectorstores.hanavector import HanaDB"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You are again connecting to our shared SAP HANA Cloud Vector Engine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# connect to HANA instance\n",
    "connection = init_env.connect_to_hana_db()\n",
    "connection.isconnected()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reference which embedding model, DB connection and table to use\n",
    "embeddings = OpenAIEmbeddings(deployment_id=variables.EMBEDDING_DEPLOYMENT_ID)\n",
    "db = HanaDB(\n",
    "    embedding=embeddings, connection=connection, table_name=variables.EMBEDDING_TABLE\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this step you are defining which LLM to use during the retrieving process. You then also assign which database to retrieve information from. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define which model to use\n",
    "chat_llm = ChatOpenAI(deployment_id=variables.LLM_DEPLOYMENT_ID)\n",
    "\n",
    "# Create a retriever instance of the vector store\n",
    "retriever = db.as_retriever(search_kwargs={\"k\": 2})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "👉 Instead of sending the query directly to the LLM, you will now create a `RetrievalQA` instance and pass both the LLM and the database to be used during the retrieval process. Once set up, you can send your query to the `Retriever`.\n",
    "\n",
    "👉 Try out different queries. Feel free to ask anything you'd like to know about the Models that are available in Generative AI Hub."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the QA instance to query llm based on custom documents\n",
    "qa = RetrievalQA.from_llm(llm=chat_llm, retriever=retriever, return_source_documents=True)\n",
    "\n",
    "# Send query\n",
    "query = \"What is data masking in the orchestration service?\"\n",
    "\n",
    "answer = qa.invoke(query)\n",
    "display(answer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for document in answer['source_documents']:\n",
    "    display(document.metadata)   \n",
    "    print(document.page_content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "👉 Go back to [05-store-embeddings-hana](05-store-embeddings-hana.ipynb) and try out different chunk sizes and/or different values for overlap. Store these chunks in a different table by adding a new variable to [variables.py](variables.py) and run this script again using the newly created table.\n",
    "\n",
    "[Next exercise](07-deploy-orchestration-service.md)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}