Skip to content

Commit 3498844

Browse files
KillianLucasKillianLucas
authored
Natural language server
Development
2 parents 6e11280 + 2e0e8bd commit 3498844

File tree

12 files changed

+1399
-841
lines changed

12 files changed

+1399
-841
lines changed

‎.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -235,3 +235,4 @@ misc/
235235

236236
# Ignore litellm_uuid.txt
237237
litellm_uuid.txt
238+
.aider*

‎README.md

-3
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,6 @@
1818
<img alt="local_explorer" src="https://github.com/OpenInterpreter/open-interpreter/assets/63927363/d941c3b4-b5ad-4642-992c-40edf31e2e7a">
1919

2020
<br>
21-
<br>
22-
<p align="center">
23-
This week we launched <strong>Local III</strong>, which introduces the Local Explorer via <strong><code>--local</code></strong>. <a href="https://changes.openinterpreter.com/log/local-iii">Read On →</a>
2421
</p>
2522
<br>
2623

‎docs/getting-started/setup.mdx

+3-3
Original file line numberDiff line numberDiff line change
@@ -65,15 +65,15 @@ To try our experimental installers, open your Terminal with admin privileges [(c
6565
<CodeGroup>
6666

6767
```bash Mac
68-
curl -sL https://raw.githubusercontent.com/OpenInterpreter/open-interpreter/main/installers/oi-mac-installer.sh | bash
68+
curl -sL https://raw.githubusercontent.com/openinterpreter/open-interpreter/main/installers/oi-mac-installer.sh | bash
6969
```
7070

7171
```powershell Windows
72-
iex "& {$(irm https://raw.githubusercontent.com/OpenInterpreter/open-interpreter/main/installers/oi-windows-installer-conda.ps1)}"
72+
iex "& {$(irm https://raw.githubusercontent.com/openinterpreter/open-interpreter/main/installers/oi-windows-installer-conda.ps1)}"
7373
```
7474

7575
```bash Linux
76-
curl -sL https://raw.githubusercontent.com/OpenInterpreter/open-interpreter/main/installers/oi-linux-installer.sh | bash
76+
curl -sL https://raw.githubusercontent.com/openinterpreter/open-interpreter/main/installers/oi-linux-installer.sh | bash
7777
```
7878

7979
</CodeGroup>

‎examples/custom_tool.ipynb

+127
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Add a Custom Tool to your Instance\n",
8+
"\n",
9+
"You can create custom tools for your instance of Open Interpreter. This is extremely helpful for adding new functionality in a reliable way.\n"
10+
]
11+
},
12+
{
13+
"cell_type": "markdown",
14+
"metadata": {},
15+
"source": [
16+
"First, create a profile and configure your instance:"
17+
]
18+
},
19+
{
20+
"cell_type": "code",
21+
"execution_count": null,
22+
"metadata": {},
23+
"outputs": [],
24+
"source": [
25+
"# Configure Open Interpreter\n",
26+
"from interpreter import interpreter\n",
27+
"\n",
28+
"interpreter.llm.model = \"claude-3-5-sonnet-20240620\"\n",
29+
"interpreter.computer.import_computer_api = True\n",
30+
"interpreter.llm.supports_functions = True\n",
31+
"interpreter.llm.supports_vision = True\n",
32+
"interpreter.llm.context_window = 100000\n",
33+
"interpreter.llm.max_tokens = 4096"
34+
]
35+
},
36+
{
37+
"cell_type": "markdown",
38+
"metadata": {},
39+
"source": [
40+
"Then you will define your custom tool by writing valid Python code within a comment. This example is for searching the AWS documentation using Perplexity:"
41+
]
42+
},
43+
{
44+
"cell_type": "code",
45+
"execution_count": null,
46+
"metadata": {},
47+
"outputs": [],
48+
"source": [
49+
"custom_tool = \"\"\"\n",
50+
"\n",
51+
"import os\n",
52+
"import requests\n",
53+
"\n",
54+
"def search_aws_docs(query):\n",
55+
"\n",
56+
" url = \"https://api.perplexity.ai/chat/completions\"\n",
57+
"\n",
58+
" payload = {\n",
59+
" \"model\": \"llama-3.1-sonar-small-128k-online\",\n",
60+
" \"messages\": [\n",
61+
" {\n",
62+
" \"role\": \"system\",\n",
63+
" \"content\": \"Be precise and concise.\"\n",
64+
" },\n",
65+
" {\n",
66+
" \"role\": \"user\",\n",
67+
" \"content\": query\n",
68+
" }\n",
69+
" ],\n",
70+
" \"temperature\": 0.2,\n",
71+
" \"top_p\": 0.9,\n",
72+
" \"return_citations\": True,\n",
73+
" \"search_domain_filter\": [\"docs.aws.amazon.com\"],\n",
74+
" \"return_images\": False,\n",
75+
" \"return_related_questions\": False,\n",
76+
" #\"search_recency_filter\": \"month\",\n",
77+
" \"top_k\": 0,\n",
78+
" \"stream\": False,\n",
79+
" \"presence_penalty\": 0,\n",
80+
" \"frequency_penalty\": 1\n",
81+
" }\n",
82+
" headers = {\n",
83+
" \"Authorization\": f\"Bearer {os.environ.get('PPLX_API_KEY')}\",\n",
84+
" \"Content-Type\": \"application/json\"\n",
85+
" }\n",
86+
"\n",
87+
" response = requests.request(\"POST\", url, json=payload, headers=headers)\n",
88+
"\n",
89+
" print(response.text)\n",
90+
"\n",
91+
" return response.text\n",
92+
"\n",
93+
"\"\"\"\n"
94+
]
95+
},
96+
{
97+
"cell_type": "markdown",
98+
"metadata": {},
99+
"source": [
100+
"Finally, you add the tool to the OI instance's computer:"
101+
]
102+
},
103+
{
104+
"cell_type": "code",
105+
"execution_count": null,
106+
"metadata": {},
107+
"outputs": [],
108+
"source": [
109+
"interpreter.computer.run(\"python\", custom_tool)"
110+
]
111+
},
112+
{
113+
"cell_type": "markdown",
114+
"metadata": {},
115+
"source": [
116+
"> Note: You can define and set multiple tools in a single instance."
117+
]
118+
}
119+
],
120+
"metadata": {
121+
"language_info": {
122+
"name": "python"
123+
}
124+
},
125+
"nbformat": 4,
126+
"nbformat_minor": 2
127+
}

‎examples/jan_computer_control.ipynb

+73
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Jan Computer Control\n",
8+
"\n",
9+
"We love Jan as an A.I. inference server. It also has a chat interface to chat with LLMs. But did you know that you can use this same chat interface as a computer control interface? Read on!\n",
10+
"\n",
11+
"[View on YouTube](https://www.youtube.com/watch?v=1l3B0AzbbjQ)"
12+
]
13+
},
14+
{
15+
"cell_type": "markdown",
16+
"metadata": {},
17+
"source": [
18+
"Install and set up Jan\n",
19+
"\n",
20+
"https://jan.ai/"
21+
]
22+
},
23+
{
24+
"cell_type": "markdown",
25+
"metadata": {},
26+
"source": [
27+
"Install Open Interpreter\n",
28+
"\n",
29+
"https://docs.openinterpreter.com/getting-started/introduction"
30+
]
31+
},
32+
{
33+
"cell_type": "markdown",
34+
"metadata": {},
35+
"source": [
36+
"Run the Open Interpreter OpenAI-compatible server.\n",
37+
"\n",
38+
"`interpreter --server`\n",
39+
"\n",
40+
"Add flags to set the `--model`, `--context_window`, or any other [setting](https://docs.openinterpreter.com/settings/all-settings) you want"
41+
]
42+
},
43+
{
44+
"cell_type": "markdown",
45+
"metadata": {},
46+
"source": [
47+
"Edit Jan's OpenAI settings to point to the local server.\n",
48+
"\n",
49+
"Settings => OpenAI => Chat Competion endpoint `http://127.0.0.1:8000/openai/chat/completions`.\n",
50+
"\n",
51+
"Jan has a requirement to set a dummy OpenAI API key."
52+
]
53+
},
54+
{
55+
"cell_type": "markdown",
56+
"metadata": {},
57+
"source": [
58+
"Go to Jan's chat window to start a new thread.\n",
59+
"\n",
60+
"Set `Model` to an OpenAI model. \n",
61+
"\n",
62+
"Start controlling your computer!"
63+
]
64+
}
65+
],
66+
"metadata": {
67+
"language_info": {
68+
"name": "python"
69+
}
70+
},
71+
"nbformat": 4,
72+
"nbformat_minor": 2
73+
}

‎examples/organize_photos.ipynb

+118
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Organize your photos with Open Interpreter"
8+
]
9+
},
10+
{
11+
"cell_type": "markdown",
12+
"metadata": {},
13+
"source": [
14+
"You can use Open Interpreter with a vision model to organize photos based on their contents. This is limited by the ability of the LLM as well as the organization of the directories storing photos. "
15+
]
16+
},
17+
{
18+
"cell_type": "markdown",
19+
"metadata": {},
20+
"source": [
21+
"> Note: It is always recommended to back up photos and files on a regular basis. Most models are intelligent enough to know the difference between `move` and `delete` but on rare occasions, files can be deleted during some operations. It is important to test on duplicated photos and to keep an eye on code written by an LLM."
22+
]
23+
},
24+
{
25+
"cell_type": "markdown",
26+
"metadata": {},
27+
"source": [
28+
"Create a profile. This example uses GPT-4o but you can use any vision model."
29+
]
30+
},
31+
{
32+
"cell_type": "code",
33+
"execution_count": null,
34+
"metadata": {},
35+
"outputs": [],
36+
"source": [
37+
"\"\"\"\n",
38+
"This is an Open Interpreter profile to organize a directory of photos. \n",
39+
"\"\"\"\n",
40+
"\n",
41+
"from interpreter import interpreter\n",
42+
"\n",
43+
"\n",
44+
"# LLM settings\n",
45+
"interpreter.llm.model = \"gpt-4o\"\n",
46+
"#interpreter.llm.model = \"ollama/codestral\"\n",
47+
"interpreter.llm.supports_vision = True\n",
48+
"interpreter.llm.execution_instructions = False\n",
49+
"interpreter.llm.max_tokens = 1000\n",
50+
"interpreter.llm.context_window = 7000\n",
51+
"interpreter.llm.load() # Loads Ollama models\n",
52+
"\n",
53+
"# Computer settings\n",
54+
"interpreter.computer.import_computer_api = True\n",
55+
"\n",
56+
"# Misc settings\n",
57+
"interpreter.auto_run = False\n",
58+
"\n"
59+
]
60+
},
61+
{
62+
"cell_type": "markdown",
63+
"metadata": {},
64+
"source": [
65+
"The following custom instruction is intended for a directory containing one sub-directory of unorganized photos and multiple empty sub-directories with names for the intended organization. Please update the custom instructions to match your use-case. This will take some trial and error, depending on the model used."
66+
]
67+
},
68+
{
69+
"cell_type": "code",
70+
"execution_count": null,
71+
"metadata": {},
72+
"outputs": [],
73+
"source": [
74+
"# Custom Instructions\n",
75+
"interpreter.custom_instructions=f\"\"\"\n",
76+
" Recap the plan before answering the user's query!\n",
77+
" Your job is to organize photos. You love organizing photos.\n",
78+
" You will be given a parent directory with sub-directories. \n",
79+
" One sub-directory will be of unorganized photos.\n",
80+
" The other sub-directories will be categories that you move the photos in to.\n",
81+
" Remember the sub-directories's names because they will be the categories for organizing.\n",
82+
" It is extremely important because these are the only options for where you move the photos.\n",
83+
" Loop through every photo in the unorganized photos directory. \n",
84+
" Skip over non-photo files by checking for common photo extensions (.jpg, .jpeg, .png, etc).\n",
85+
" In this loop you will determine the description of each image one at a time. \n",
86+
" Use `computer.vision.query()` to get a description of the image.\n",
87+
" `computer.vision.query()` takes a `path=` argument to know which photo to describe. \n",
88+
" Print out the description so you can get the full context.\n",
89+
" Determine which sub-directory the photo should go in to.\n",
90+
" Every photo needs to go into one of the sub-directories.\n",
91+
" Make sure you actually move the photo. \n",
92+
" Your task is done when every photo in the unorganized photos directory has been moved to another directory. \n",
93+
" **Confirm that the unorganized photos directory has no more photos**.\n",
94+
" \"\"\"\n"
95+
]
96+
},
97+
{
98+
"cell_type": "markdown",
99+
"metadata": {},
100+
"source": [
101+
"Save the profile with a descriptive name. Then run interpreter with:\n",
102+
"\n",
103+
"`interpreter --profile <profileName.py>`\n",
104+
"\n",
105+
"Then ask it to organize the directory:\n",
106+
"\n",
107+
"`Please organize this directory: /path/to/directory`"
108+
]
109+
}
110+
],
111+
"metadata": {
112+
"language_info": {
113+
"name": "python"
114+
}
115+
},
116+
"nbformat": 4,
117+
"nbformat_minor": 2
118+
}

0 commit comments

Comments
 (0)