You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
An open-source Swift package that simplifies LLM message completions, inspired by [liteLLM](https://litellm.ai/) and adapted for Swift developers, following Swift conventions.
13
12
13
+
## Description
14
+
15
+
Call different LLM APIs using the OpenAI format; currently supporting [OpenAI](https://github.com/jamesrochabrun/SwiftOpenAI), [Anthropic](https://github.com/jamesrochabrun/SwiftAnthropic), [Gemini](https://github.com/google-gemini/generative-ai-swift).
16
+
17
+
Also call any local model using [Ollama OpenAI compatibility endopints](https://ollama.com/blog/openai-compatibility). You can use models like [llama3](https://ollama.com/library/llama3) or [mistral](https://ollama.com/library/mistral).
18
+
14
19
## Table of Contents
15
20
16
-
-[Description](#description)
17
21
-[Installation](#installation)
18
22
-[Usage](#usage)
19
23
-[Message](#message)
20
24
-[Collaboration](#collaboration)
21
25
-[OpenAI Azure](#openAI-azure)
22
26
-[OpenAI AIProxy](#openai-aiproxy)
23
-
24
-
## Description
25
-
26
-
Call different LLM APIs using the OpenAI format; currently supporting [OpenAI](https://github.com/jamesrochabrun/SwiftOpenAI), [Anthropic](https://github.com/jamesrochabrun/SwiftAnthropic), and [Gemini](https://github.com/google-gemini/generative-ai-swift).
27
+
-[Ollama](#ollama)
27
28
28
29
## Installation
29
30
@@ -54,13 +55,16 @@ First, import the PolyAI package:
54
55
importPolyAI
55
56
```
56
57
57
-
Then, define the LLM configurations. Currently, OpenAI, Anthropic and Gemini are supported:
58
+
Then, define the LLM configurations.
59
+
Currently, OpenAI, Anthropic and Gemini are supported, you can also use Ollama or any provider that provides local models with OpenAI endpoint compatibilities to use local models such llama3 or Mistral.
58
60
59
61
```swift
60
62
let openAIConfiguration: LLMConfiguration = .openAI(.api(key: "your_openai_api_key_here"))
61
63
let anthropicConfiguration: LLMConfiguration = .anthropic(apiKey: "your_anthropic_api_key_here")
62
64
let geminiConfiguration: LLMConfiguration = .gemini(apiKey: "your_gemini_api_key_here")
63
-
let configurations = [openAIConfiguration, anthropicConfiguration, geminiConfiguration]
65
+
let ollamaConfiguration: LLMConfiguration = .ollama(url: "http://localhost:11434")
66
+
67
+
let configurations = [openAIConfiguration, anthropicConfiguration, geminiConfiguration, ollamaConfiguration]
64
68
```
65
69
66
70
With the configurations set, initialize the service:
@@ -69,7 +73,7 @@ With the configurations set, initialize the service:
69
73
let service = PolyAIServiceFactory.serviceWith(configurations)
70
74
```
71
75
72
-
Now, you have access to OpenAI, Anthropic and Gemini APIs in a single package. 🚀
76
+
Now, you have access to OpenAI, Anthropic, Gemini, llama3, Mistral models in a single package. 🚀
73
77
74
78
## Message
75
79
@@ -93,7 +97,15 @@ To interact with Gemini instead, all you need to do (again) is change just one l
let stream = try await service.streamMessage(parameters)
98
110
```
99
111
@@ -117,6 +129,23 @@ let aiProxyConfiguration: LLMConfiguration = .openAI(.aiProxy(aiproxyPartialKey:
117
129
118
130
More information can be found [here](https://github.com/jamesrochabrun/SwiftOpenAI?tab=readme-ov-file#aiproxy).
119
131
132
+
### Ollama
133
+
134
+
To interact with local models using [Ollama OpenAI compatibility endpoints](https://ollama.com/blog/openai-compatibility), use the following configuration setup.
135
+
136
+
1 - Download [Ollama](https://ollama.com/) if yo don't have it installed already.
137
+
2 - Download the model you need, e.g for `llama3` type in terminal:
138
+
```
139
+
ollama pull llama3
140
+
```
141
+
142
+
Once you have the model installed locally you are ready to use PolyAI!
143
+
144
+
```swift
145
+
let ollamaConfiguration: LLMConfiguration = .ollama(url: "http://localhost:11434")
146
+
```
147
+
More information can be found [here](https://github.com/jamesrochabrun/SwiftOpenAI?tab=readme-ov-file#ollama).
148
+
120
149
## Collaboration
121
150
122
151
Open a PR forany proposed change pointing it to `main` branch.
0 commit comments