Update the docs for local models with ollama (#4)

Author: Binlogo

docs/blog/why-aiscript.mdx

@@ -191,6 +191,12 @@ api_key = "YOUR_API_KEY"
 
 [ai.anthropic]
 api_key = "YOUR_API_KEY"
+
+# Local models with Ollama (no API key needed)
+[ai.ollama]
+api_endpoint = "http://localhost:11434/v1"
+model = "llama3.2"
+api_key = ""
 ```
 </Tab>
 </Tabs>

docs/guide/ai/_meta.json

@@ -1 +1 @@
-["prompt", "function", "agent"]
+["prompt", "function", "agent", "local-models"]

docs/guide/ai/local-models.mdx

@@ -0,0 +1,120 @@
+import { Badge, Tab, Tabs } from "rspress/theme";
+
+# Local Models with Ollama
+
+AIScript supports running AI models locally through [Ollama](https://ollama.ai/), providing privacy, cost-effectiveness, and offline capabilities.
+
+## Setting Up Ollama
+
+### Installation
+
+1. **Install Ollama** from [ollama.ai](https://ollama.ai/)
+2. **Start Ollama** service (runs on port 11434 by default)
+3. **Pull models** you want to use:
+
+```bash
+# Popular general-purpose models
+ollama pull llama3.2
+
+# Coding models
+ollama pull codellama
+ollama pull deepseek-coder
+
+# Specialized models
+ollama pull deepseek-r1
+ollama pull gemma2
+ollama pull qwen2.5
+```
+
+### Verification
+
+Check that Ollama is running and models are available:
+
+```bash
+# List installed models
+ollama list
+
+# Test a model
+ollama run llama3.2 "Hello, how are you?"
+```
+
+## Configuration
+
+Configure AIScript to use Ollama models:
+
+<Tabs>
+<Tab label="project.toml">
+
+```toml
+[ai.ollama]
+api_endpoint = "http://localhost:11434/v1"
+model = "llama3.2"  # Default model
+```
+
+</Tab>
+<Tab label="Environment Variable">
+
+```bash
+export OLLAMA_API_ENDPOINT="http://localhost:11434/v1"
+```
+
+</Tab>
+</Tabs>
+
+## Basic Usage
+
+### Simple Prompts
+
+```rust
+let response = prompt {
+    input: "Explain how neural networks work",
+    model: "llama3.2"
+};
+```
+
+## Troubleshooting
+
+### Common Issues
+
+**Ollama not responding:**
+```bash
+# Check if Ollama is running
+curl http://localhost:11434/api/tags
+
+# Restart Ollama service
+ollama serve
+```
+
+**Model not found:**
+```bash
+# List available models
+ollama list
+
+# Pull missing model
+ollama pull llama3.2
+```
+
+**Out of memory:**
+- Try a smaller model (e.g., `phi3` instead of `llama3.1`)
+- Close other applications
+- Use quantized models
+
+### Performance Issues
+
+- **Slow responses**: Try smaller models or increase hardware resources
+- **High memory usage**: Monitor with `htop` or Activity Monitor
+- **GPU not utilized**: Ensure GPU drivers are properly installed
+
+## Comparison: Local vs Cloud Models
+
+| Aspect | Local Models | Cloud Models |
+|--------|-------------|--------------|
+| **Privacy** | ✅ Complete | ❌ Data sent externally |
+| **Cost** | ✅ One-time setup | ❌ Per-token billing |
+| **Internet** | ✅ Works offline | ❌ Requires connection |
+| **Latency** | ✅ Low (local) | ❌ Network dependent |
+| **Quality** | ⚠️ Model dependent | ✅ Usually higher |
+| **Maintenance** | ❌ Self-managed | ✅ Fully managed |
+| **Scaling** | ❌ Hardware limited | ✅ Unlimited |
+
+Local models with Ollama provide a powerful way to integrate AI capabilities into your AIScript applications while maintaining full control over your data and costs.

docs/guide/ai/prompt.mdx

@@ -51,7 +51,7 @@ let response = prompt f"Explain {topic} at a {detail_level} level";
 
 ## API Keys and Configuration
 
-To use the `prompt` feature, you need to configure your API keys. AIScript supports multiple LLM providers.
+To use the `prompt` feature, you need to configure your API keys. AIScript supports multiple LLM providers including cloud-based services and local models.
 
 ### Environment Variables
 
@@ -63,6 +63,9 @@ export OPENAI_API_KEY="your_openai_key"
 
 # For Anthropic Claude
 export CLAUDE_API_KEY="your_claude_key"
+
+# For Ollama (local models)
+export OLLAMA_API_ENDPOINT="http://localhost:11434/v1"
 ```
 
 ### Configuration File
@@ -78,10 +81,47 @@ api_key = "YOUR_OPENAI_API_KEY"
 
 [ai.anthropic]
 api_key = "YOUR_CLAUDE_API_KEY"
+
+# For Ollama (local models)
+[ai.ollama]
+api_endpoint = "http://localhost:11434/v1"
+model = "llama3.2" # or any other model installed in your Ollama instance
 ```
 </Tab>
 </Tabs>
 
+## Local Models with Ollama
+
+AIScript supports running local models through [Ollama](https://ollama.ai/), which allows you to run AI models on your own hardware without sending data to external services.
+
+### Setting Up Ollama
+
+1. **Install Ollama** from [ollama.ai](https://ollama.ai/)
+2. **Pull models** you want to use:
+   ```bash
+   ollama pull llama3.2
+   ollama pull deepseek-r1
+   ollama pull codellama
+   ```
+3. **Start Ollama** (it runs on port 11434 by default)
+4. **Configure AIScript** to use Ollama
+
+### Example with Local Models
+
+```rust
+// Using a local Llama model
+let response = prompt {
+    input: "What is rust?",
+    model: "llama3.2"
+};
+
+// Using a local coding model
+let code_explanation = prompt {
+    input: "Review this Python function for potential issues",
+    model: "codellama"
+};
+```
+
 ## Error Handling <Badge text="Not supported yet" type="warning" />
 
 Like other AIScript operations, you can use error handling with the `prompt` keyword:
@@ -135,6 +175,12 @@ let claude_response = prompt {
     input: f"Summarize this article: {article_text}" ,
     model: "claude-3.7"
 };
+
+// Using Ollama (local models)
+let local_response = prompt {
+    input: f"Summarize this article: {article_text}",
+    model: "llama3.2"
+};
 ```
 
 ### Chaining Prompts

docs/guide/getting-started/introduction.md

@@ -78,6 +78,21 @@ You can open [http://localhost:8080/redoc](http://localhost:8080/redoc) to see t
 
 ![](/guide/open-api.png)
 
+### Local Models Support
+
+AIScript also supports local models through [Ollama](https://ollama.ai/), allowing you to run AI models on your own hardware:
+
+```javascript
+$ ollama pull llama3.2
+$ export OLLAMA_API_ENDPOINT=http://localhost:11434/v1
+$ cat local.ai
+let a = prompt {
+    input: "What is rust?",
+    model: "llama3.2"
+};
+print(a);
+```
+
 ## Use Cases
 
 AIScript excels in these scenarios:

docs/reference/configuration/ai.md

@@ -2,6 +2,15 @@
 
 > For AI observability, please refer to [observability](./observability.md).
 
+AIScript supports multiple AI providers:
+
+- **OpenAI**: Cloud-based models including GPT-4 and GPT-3.5
+- **DeepSeek**: Efficient and powerful models 
+- **Anthropic**: Claude models for advanced reasoning
+- **Ollama**: 100+ local models from various providers running on your own hardware
+
+## Configuration
+
 ```toml
 [ai.openai]
 api_key = "YOUR_API_KEY"
@@ -20,4 +29,35 @@ api_key = "YOUR_API_KEY"
 # optional, default is the official Anthropic API endpoint
 api_endpoint = "YOUR_API_ENDPOINT"
 model = "claude-3-5-sonnet-latest"
+
+# Ollama - Local models
+[ai.ollama]
+api_endpoint = "http://localhost:11434/v1"  # Default Ollama endpoint
+model = "llama3.2"  # or any other model installed in your Ollama instance
+```
+
+## Using Ollama
+
+[Ollama](https://ollama.ai/) allows you to run local AI models on your own hardware. To use Ollama with AIScript:
+
+1. **Install Ollama** from [ollama.ai](https://ollama.ai/)
+2. **Pull your desired models** (e.g., `ollama pull llama3.2`)
+3. **Make sure Ollama is running** locally
+4. **Configure AIScript** to use Ollama as shown above or by setting the `OLLAMA_API_ENDPOINT` environment variable
+
+### Available Models
+
+Ollama provides access to 100+ models ranging from small 135M parameter models to massive 671B parameter models, including:
+
+- **Llama family**: `llama3.2`, `llama3.1`, `codellama`
+- **DeepSeek models**: `deepseek-r1`, `deepseek-v3`, `deepseek-coder`
+- **Specialized models**: `phi3`, `gemma2`, `qwen2.5`, `mistral`
+- **And many more**: See the [full list of available models](https://ollama.com/search)
+
+### Environment Variables
+
+You can also configure Ollama using environment variables:
+
+```bash
+export OLLAMA_API_ENDPOINT="http://localhost:11434/v1"
 ```