|
19 | 19 | <table> |
20 | 20 | <thead> |
21 | 21 | <tr> |
22 | | - <th>GPU Brand</th> |
23 | | - <th>Windows Support</th> |
24 | | - <th>Linux Support</th> |
| 22 | + <th>GPU</th> |
| 23 | + <th>Windows</th> |
| 24 | + <th>Linux</th> |
| 25 | + <th>Requirements</th> |
25 | 26 | </tr> |
26 | 27 | </thead> |
27 | 28 | <tbody> |
28 | 29 | <tr> |
29 | 30 | <td>Nvidia</td> |
30 | | - <td>✅ (requires CUDA 11.7 or 11.8)</td> |
31 | | - <td>✅ (requires CUDA 11.7 or 11.8)</td> |
| 31 | + <td>✅</td> |
| 32 | + <td>✅</td> |
| 33 | + <td>CUDA 11.8 or 11.7</td> |
32 | 34 | </tr> |
33 | 35 | <tr> |
34 | 36 | <td>AMD</td> |
35 | 37 | <td>❌</td> |
36 | | - <td>✅ (requires ROCm 5.4.2)</td> |
| 38 | + <td>✅</td> |
| 39 | + <td>ROCm 5.4.2</td> |
37 | 40 | </tr> |
38 | 41 | <tr> |
39 | 42 | <td>Apple/Metal</td> |
40 | | - <td colspan="2" align="center"> ✅ (requires "Xcode Command Line Tools")</td> |
| 43 | + <td colspan="3" align="center"> ✅ ("Xcode Command Line Tools")</td> |
41 | 44 | </tr> |
42 | 45 | </tbody> |
43 | 46 | </table> |
44 | 47 | </div> |
45 | 48 |
|
46 | | - |
47 | 49 | <!-- Table of Contents --> |
48 | 50 |
|
49 | 51 | <div align="center"> |
|
52 | 54 |
|
53 | 55 | <div align="center"> |
54 | 56 | <a href="#installation">Installation</a> | |
55 | | - <a href="#usage-guide">Usage</a> | |
| 57 | + <a href="#usage">Usage</a> | |
56 | 58 | <a href="#contact">Contact Me</a> |
57 | 59 | </div> |
58 | 60 |
|
59 | 61 | ## Installation |
60 | 62 |
|
61 | | -* **Step 1**: Install the appropriate software if you intend to use GPU-acceleration: |
62 | | - |
| 63 | +* **Step 1** ➜ Install the appropriate software if you intend to use GPU-acceleration: |
63 | 64 | * **For NVIDIA GPUs** ➜ install [CUDA 11.8](https://developer.nvidia.com/cuda-11-8-0-download-archive) or [CUDA 11.7](https://developer.nvidia.com/cuda-11-7-0-download-archive) for your specific operating system. |
64 | 65 | * **For AMD GPUs** ➜ install ROCm version 5.4.2; instructions are [HERE](https://rocmdocs.amd.com/en/latest/deploy/linux/quick_start.html) and [HERE](https://rocmdocs.amd.com/en/latest/deploy/linux/index.html). |
65 | 66 | * Unfortunately, gpu-accleration will [only work on Linux systems](https://github.com/RadeonOpenCompute/ROCm/blob/develop/docs/rocm.md). |
66 | 67 | * **For Apple/Metal/MPS** ➜ install [Xcode Command Line Tools](https://www.makeuseof.com/install-xcode-command-line-tools/). |
67 | | -* **Step 2**: Download or clone this repository to a directory on your computer. |
68 | | -* **Step 3**: Open a command prompt from within the directory and create a virtual environment: |
| 68 | +* **Step 2** ➜ Download or clone this repository to a directory on your computer. |
| 69 | +* **Step 3** ➜ Open a command prompt from within the directory and create a virtual environment: |
69 | 70 | ``` |
70 | 71 | python -m venv . |
71 | 72 | ``` |
72 | | -* **Step 4**: Activate the virtual environment: |
| 73 | +* **Step 4** ➜ Activate the virtual environment: |
73 | 74 | ``` |
74 | 75 | .\Scripts\activate |
75 | 76 | ``` |
76 | | -* **Step 5**: Update [PIP](https://pip.pypa.io/en/stable/index.html): |
| 77 | +* **Step 5** ➜ Update [PIP](https://pip.pypa.io/en/stable/index.html): |
77 | 78 | ``` |
78 | 79 | python -m pip install --upgrade pip |
79 | 80 | ``` |
80 | | -* **Step 6**: Install PyTorch with the appropriate "build:" |
81 | | - |
82 | | - * **Windows/CUDA 11.8:** ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118``` |
83 | | - * **Windows/CUDA 11.7:** ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117``` |
84 | | - * **Windows/CPU-only:** ```pip install torch torchvision torchaudio``` |
85 | | - * **Linux/CUDA 11.8:** ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118``` |
86 | | - * **Linux/CUDA 11.7:** ```pip install torch torchvision torchaudio``` |
87 | | - * **Linux/ROCm 5.4.2:** ```install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm5.4.2``` |
88 | | - * **Linux/CPU-only:** ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu``` |
89 | | - * **Apple/Metal/MPS:** ```pip pip install torch torchvision torchaudio``` |
| 81 | +* **Step 6** ➜ Install PyTorch with the appropriate "build:" |
| 82 | + |
| 83 | + * **Windows/CUDA 11.8** ➜ ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118``` |
| 84 | + * **Windows/CUDA 11.7** ➜ ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117``` |
| 85 | + * **Windows/CPU-only** ➜ ```pip install torch torchvision torchaudio``` |
| 86 | + * **Linux/CUDA 11.8** ➜ ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118``` |
| 87 | + * **Linux/CUDA 11.7** ➜ ```pip install torch torchvision torchaudio``` |
| 88 | + * **Linux/ROCm 5.4.2** ➜ ```install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm5.4.2``` |
| 89 | + * **Linux/CPU-only** ➜ ```pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu``` |
| 90 | + * **Apple/Metal/MPS** ➜ ```pip pip install torch torchvision torchaudio``` |
90 | 91 | * **Metal/MPS** [speedup comparison](https://explosion.ai/blog/metal-performance-shaders) to a 5950x and RTX 3090. |
91 | 92 |
|
92 | | -* **Step 7:** Doublecheck that you installed gpu-acceleration properly: |
| 93 | +* **Step 7** ➜ Doublecheck that you installed gpu-acceleration properly: |
93 | 94 | ``` |
94 | 95 | python check_gpu.py |
95 | 96 | ``` |
96 | 97 |
|
97 | | -* **Step 8**: Install the dependencies listed in [requirements.txt](https://github.com/MicrosoftDocs/visualstudio-docs/blob/main/docs/python/managing-required-packages-with-requirements-txt.md): |
| 98 | +* **Step 8** ➜ Install the dependencies listed in [requirements.txt](https://github.com/MicrosoftDocs/visualstudio-docs/blob/main/docs/python/managing-required-packages-with-requirements-txt.md): |
98 | 99 | ``` |
99 | 100 | pip install -r requirements.txt |
100 | 101 | ``` |
101 | | -* **Step 9**: |
102 | | - * Lastly, you must install the appropriate version of Git (https://git-scm.com/downloads). |
| 102 | +* **Step 9** ➜ Lastly, you must install the appropriate version of Git (https://git-scm.com/downloads). |
103 | 103 |
|
104 | 104 | [Back to top](#top) |
105 | 105 |
|
106 | | -## Usage Guide |
| 106 | +## Usage |
107 | 107 |
|
108 | 108 | * **Step 1**: Open a command prompt in the directory of my scripts, activate the virtual environment, and run: |
109 | 109 | ``` |
110 | 110 | python gui.py |
111 | 111 | ``` |
112 | | -* **Step 2**: Click "Download Embedding Model" and download a model. The GUI will hang. Wait. Proceed to the next step. |
113 | | -* **Step 3**: Click "Select Embedding Model Directory" and select the DIRECTORY containing the model you want to use. |
114 | | -* **Step 4**: Click "Choose Documents for Database" and choose one or more PDF files to put in the database. |
| 112 | +* **Step 2** ➜ Click "Download Embedding Model" and download a model. The GUI will hang. Wait. Proceed to the next step. |
| 113 | +* **Step 3** ➜ Click "Select Embedding Model Directory" and select the DIRECTORY containing the model you want to use. |
| 114 | +* **Step 4** ➜ Click "Choose Documents for Database" and choose one or more PDF files to put in the database. |
115 | 115 | * Must have had OCR done on them or have text in them already. If you can select the text when viewing it, you're good. |
116 | | -* **Step 5**: Click "Create Vector Database." The GUI will hang. Watch "CUDA" usage. When CUDA drops to zero, proceed to the next step. |
117 | | -* **Step 6**: Open up LM Studio and load a model (only Llama2-based models currently work). |
118 | | -* **Step 7**: Click "Start Server" within LM Studio, enter your question, and click "Submit Question." |
| 116 | +* **Step 5** ➜ Click "Create Vector Database." The GUI will hang. Watch "CUDA" usage. When CUDA drops to zero, proceed to the next step. |
| 117 | +* **Step 6** ➜ Open up LM Studio and load a model (only Llama2-based models currently work). |
| 118 | +* **Step 7** ➜ Click "Start Server" within LM Studio, enter your question, and click "Submit Question." |
119 | 119 |
|
120 | 120 |
|
121 | 121 | [Back to top](#top) |
122 | 122 |
|
123 | 123 | ## Contact |
124 | 124 |
|
125 | | -All suggestions (positive and negative) are welcome. I can be reached at "bbc@chintellalaw.com" or feel free to message me on the [LM Studio Discord Server](https://discord.gg/aPQfnNkxGC). |
| 125 | +All suggestions (positive and negative) are welcome. "bbc@chintellalaw.com" or feel free to message me on the [LM Studio Discord Server](https://discord.gg/aPQfnNkxGC). |
126 | 126 |
|
127 | 127 | [Back to top](#top) |
128 | 128 |
|
|
0 commit comments