You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: examples/README.md
+10-3
Original file line number
Diff line number
Diff line change
@@ -8,39 +8,46 @@ Please send a pull request if you find things that belongs to here.
8
8
Note that all examples below run in-browser and use WebGPU as a backend.
9
9
10
10
#### Project List
11
+
11
12
-[get-started](get-started): minimum get started example with chat completion.
12
13
13
14
[](https://jsfiddle.net/neetnestor/yac9gbwf/)
14
15
[](https://codepen.io/neetnestor/pen/NWVdgey)
16
+
15
17
-[simple-chat-js](simple-chat-js): a mininum and complete chat bot app in vanilla JavaScript.
16
18
17
19
[](https://jsfiddle.net/neetnestor/4nmgvsa2/)
18
20
[](https://codepen.io/neetnestor/pen/vYwgZaG)
21
+
19
22
-[simple-chat-ts](simple-chat-ts): a mininum and complete chat bot app in TypeScript.
20
23
-[get-started-web-worker](get-started-web-worker): same as get-started, but using web worker.
21
24
-[next-simple-chat](next-simple-chat): a mininum and complete chat bot app with [Next.js](https://nextjs.org/).
22
25
-[multi-round-chat](multi-round-chat): while APIs are functional, we internally optimize so that multi round chat usage can reuse KV cache
23
26
24
27
#### Advanced OpenAI API Capabilities
28
+
25
29
These examples demonstrate various capabilities via WebLLM's OpenAI-like API.
30
+
26
31
-[streaming](streaming): return output as chunks in real-time in the form of an AsyncGenerator
27
32
-[json-mode](json-mode): efficiently ensure output is in json format, see [OpenAI Reference](https://platform.openai.com/docs/guides/text-generation/chat-completions-api) for more.
28
33
-[json-schema](json-schema): besides guaranteeing output to be in JSON, ensure output to adhere to a specific JSON schema specified the user
29
34
-[function-calling](function-calling) (WIP): function calling with fields `tools` and `tool_choice`.
30
35
-[seed-to-reproduce](seed-to-reproduce): use seeding to ensure reproducible output with fields `seed`.
31
36
32
37
#### Chrome Extension
38
+
33
39
-[chrome-extension](chrome-extension): chrome extension that does not have a persistent background
34
40
-[chrome-extension-webgpu-service-worker](chrome-extension-webgpu-service-worker): chrome extension using service worker, hence having a persistent background
35
41
36
42
#### Others
43
+
37
44
-[logit-processor](logit-processor): while `logit_bias` is supported, we additionally support stateful logit processing where users can specify their own rules. We also expose low-level API `forwardTokensAndSample()`.
38
45
-[cache-usage](cache-usage): demonstrates how WebLLM supports both the [Cache API](https://developer.mozilla.org/en-US/docs/Web/API/Cache) and [IndexedDB cache](https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API), and
39
-
users can pick with `appConfig.useIndexedDBCache`. Also demonstrates various cache utils such as checking
40
-
whether a model is cached, deleting a model's weights from cache, deleting a model library wasm from cache, etc.
46
+
users can pick with `appConfig.useIndexedDBCache`. Also demonstrates various cache utils such as checking
47
+
whether a model is cached, deleting a model's weights from cache, deleting a model library wasm from cache, etc.
41
48
-[simple-chat-upload](simple-chat-upload): demonstrates how to upload local models to WebLLM instead of downloading via a URL link
42
49
43
50
## Demo Spaces
44
51
45
-
-[web-llm-embed](https://huggingface.co/spaces/matthoffner/web-llm-embed): document chat prototype using react-llm with transformers.js embeddings
52
+
-[web-llm-embed](https://huggingface.co/spaces/matthoffner/web-llm-embed): document chat prototype using react-llm with transformers.js embeddings
46
53
-[DeVinci](https://x6occ-biaaa-aaaai-acqzq-cai.icp0.io/): AI chat app based on WebLLM and hosted on decentralized cloud platform
Copy file name to clipboardExpand all lines: examples/cache-usage/README.md
+1-2
Original file line number
Diff line number
Diff line change
@@ -6,10 +6,9 @@ demonstrate the utility cache functions such as deleting models, checking if mod
6
6
7
7
For more information about the two caches, see: https://developer.mozilla.org/en-US/docs/Web/API/Storage_API/Storage_quotas_and_eviction_criteria#what_technologies_store_data_in_the_browser.
8
8
9
-
To inspect the downloaded artifacts in your browser, open up developer console, go to application,
9
+
To inspect the downloaded artifacts in your browser, open up developer console, go to application,
10
10
and you will find the artifacts under either `IndexedDB` or `Cache storage`.
11
11
12
-
13
12
To run the exapmle, you can do the following steps under this folder
> Service worker support in WebGPU is enabled by default in [Chrome 124](https://chromiumdash.appspot.com/commit/8d78510e4aca5ac3cd8ee4a33e96b404eaa43246).
8
7
> If you are using Chrome 123, go to `chrome://flags/#enable-experimental-web-platform-features`, enable the `#enable-experimental-web-platform-features` flag, and **relaunch the browser**.
9
8
10
9
This example shows how we can create a Chrome extension using WebGPU and service worker.
11
10
12
11
- The project structure is as follows:
13
-
-`manifest.json`: A required file that lists important information about the structure and behavior of that extension. Here we are using manifest V3.
14
-
-`popup.ts`: Script of the extension pop-up window.
15
-
-`background.ts`: Script of the service worker. An extension service worker is loaded when it is needed, and unloaded when it goes dormant.
16
-
-`content.js`: Content script that interacts with DOM.
12
+
-`manifest.json`: A required file that lists important information about the structure and behavior of that extension. Here we are using manifest V3.
13
+
-`popup.ts`: Script of the extension pop-up window.
14
+
-`background.ts`: Script of the service worker. An extension service worker is loaded when it is needed, and unloaded when it goes dormant.
15
+
-`content.js`: Content script that interacts with DOM.
17
16
- Run
17
+
18
18
```bash
19
19
npm install
20
20
npm run build
21
21
```
22
22
23
23
This will create a new directory at `./dist/`. To load the extension into Chrome, go to Extensions > Manage Extensions and select Load Unpacked. Add the `./dist/` directory. You can now pin the extension to your toolbar and use it to chat with your favorite model!
24
24
25
-
**Note**: This example disables chatting using the contents of the active tab by default.
25
+
**Note**: This example disables chatting using the contents of the active tab by default.
26
26
To enable it, set `useContext` in `popup.ts` to `true`. More info about this feature can be found
In the year 2154, humanity had colonized several planets in the distant reaches of the galaxy. The planet of Xylophia-IV was one of the most remote and inhospitable, with temperatures often dropping to -200 degrees Celsius. Despite these harsh conditions, a team of scientists had established a research station on the planet to study the unique geological formations and exotic flora and fauna.
2
-
3
-
One day, while conducting a routine survey of the planet's surface, the team discovered an strange object buried deep in the ice. As they examined it closer, they realized it was a small, metallic capsule with a glowing blue symbol etched onto its surface.
4
-
5
-
The team's leader, a brilliant scientist named Dr. Maria Rodriguez, was immediately intrigued by the capsule's mysterious origins. She ordered her team to bring it back to the research station for further analysis.
6
-
7
-
After weeks of studying the capsule, the team finally cracked the code to the symbol etched onto its surface. It was a message from an alien race, warning Earth of an impending attack from an unknown threat.
8
-
9
-
The team was shocked and dismayed by the news, but they knew they had to act quickly to warn the rest of humanity. They transmitted the message to the nearest space station, which relayed it to Earth's government.
10
-
11
-
As the threat of attack loomed near, the team remained on high alert, ready to face whatever dangers lay ahead. They had uncovered a secrets of the universe, and now they were determined to protect their planet and its inhabitants at all costs.
1
+
In the year 2154, humanity had colonized several planets in the distant reaches
2
+
of the galaxy. The planet of Xylophia-IV was one of the most remote and
3
+
inhospitable, with temperatures often dropping to -200 degrees Celsius. Despite
4
+
these harsh conditions, a team of scientists had established a research station
5
+
on the planet to study the unique geological formations and exotic flora and
6
+
fauna. One day, while conducting a routine survey of the planet's surface, the
7
+
team discovered an strange object buried deep in the ice. As they examined it
8
+
closer, they realized it was a small, metallic capsule with a glowing blue
9
+
symbol etched onto its surface. The team's leader, a brilliant scientist named
10
+
Dr. Maria Rodriguez, was immediately intrigued by the capsule's mysterious
11
+
origins. She ordered her team to bring it back to the research station for
12
+
further analysis. After weeks of studying the capsule, the team finally cracked
13
+
the code to the symbol etched onto its surface. It was a message from an alien
14
+
race, warning Earth of an impending attack from an unknown threat. The team was
15
+
shocked and dismayed by the news, but they knew they had to act quickly to warn
16
+
the rest of humanity. They transmitted the message to the nearest space station,
17
+
which relayed it to Earth's government. As the threat of attack loomed near, the
18
+
team remained on high alert, ready to face whatever dangers lay ahead. They had
19
+
uncovered a secrets of the universe, and now they were determined to protect
0 commit comments