From 34f8b558654487100d2f44934750501165105e7e Mon Sep 17 00:00:00 2001 From: aevesdocker Date: Tue, 22 Apr 2025 11:22:54 +0100 Subject: [PATCH 1/8] ENGDOCS-2572 --- .../manuals/desktop/features/gordon/_index.md | 2 ++ .../manuals/desktop/features/model-runner.md | 17 ++++++++++++++++- data/summary.yaml | 2 +- 3 files changed, 19 insertions(+), 2 deletions(-) diff --git a/content/manuals/desktop/features/gordon/_index.md b/content/manuals/desktop/features/gordon/_index.md index 22f15e89738..4c4295ab8ab 100644 --- a/content/manuals/desktop/features/gordon/_index.md +++ b/content/manuals/desktop/features/gordon/_index.md @@ -97,6 +97,8 @@ If you have concerns about data collection or usage, you can 9. Select **Apply & restart**. +You can also enable Ask Gordon from the **Ask Gordon** tab if you have selected the **Access experimental features** setting. Simply select the **Enable Ask Gordon** button, and then accept the Docker AI terms of service agreement. + ## Using Ask Gordon The primary interfaces to Docker's AI capabilities are through the **Ask diff --git a/content/manuals/desktop/features/model-runner.md b/content/manuals/desktop/features/model-runner.md index daec14150b4..0772f2eb3f9 100644 --- a/content/manuals/desktop/features/model-runner.md +++ b/content/manuals/desktop/features/model-runner.md @@ -17,7 +17,8 @@ The Docker Model Runner plugin lets you: - [Pull models from Docker Hub](https://hub.docker.com/u/ai) - Run AI models directly from the command line - Manage local models (add, list, remove) -- Interact with models using a submitted prompt or in chat mode +- Interact with models using a submitted prompt or in chat mode in the CLI or Docker Desktop Dashboard +- Push models to Docker Hub Models are pulled from Docker Hub the first time they're used and stored locally. They're loaded into memory only at runtime when a request is made, and unloaded when not in use to optimize resources. Since models can be large, the initial pull may take some time — but after that, they're cached locally for faster access. You can interact with the model using [OpenAI-compatible APIs](#what-api-endpoints-are-available). @@ -31,6 +32,8 @@ Models are pulled from Docker Hub the first time they're used and stored locally 6. Navigate to **Features in development**. 7. From the **Beta** tab, check the **Enable Docker Model Runner** setting. +You can now use the `docker model` command in the CLI and view and interact with your local models in the **Models** tab in the Docker Desktop Dashboard. + ## Available commands ### Model runner status @@ -84,6 +87,8 @@ Downloaded: 257.71 MB Model ai/smollm2 pulled successfully ``` +The models also display in the Docker Desktop Dashboard. + ### List available models Lists all models currently pulled to your local environment. @@ -131,6 +136,16 @@ Hi there! It's SmolLM, AI assistant. How can I help you today? Chat session ended. ``` +> [!TIP] +> +> You can also use chat mode in the Docker Desktop Dashboard when you select the model in the **Models** tab. + +### Push a model + +```console +docker model push / +``` + ### Remove a model Removes a downloaded model from your system. diff --git a/data/summary.yaml b/data/summary.yaml index 4fe1f4f0860..07ce36121d0 100644 --- a/data/summary.yaml +++ b/data/summary.yaml @@ -145,7 +145,7 @@ Docker GitHub Copilot: Docker Model Runner: availability: Beta requires: Docker Desktop 4.40 and later - for: Docker Desktop for Mac with Apple Silicon + for: Docker Desktop for Mac with Apple Silicon or Windows with NVIDIA GPUs Docker Projects: availability: Beta Docker Init: From 058847ac0593e97f88f28928a5568f0d9278431c Mon Sep 17 00:00:00 2001 From: aevesdocker Date: Tue, 22 Apr 2025 13:22:45 +0100 Subject: [PATCH 2/8] logs command --- .../manuals/desktop/features/model-runner.md | 21 ++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/content/manuals/desktop/features/model-runner.md b/content/manuals/desktop/features/model-runner.md index 0772f2eb3f9..a8934645234 100644 --- a/content/manuals/desktop/features/model-runner.md +++ b/content/manuals/desktop/features/model-runner.md @@ -123,7 +123,7 @@ Hello! How can I assist you today? #### Interactive chat ```console -docker model run ai/smollm2 +$ docker model run ai/smollm2 ``` Output: @@ -140,12 +140,27 @@ Chat session ended. > > You can also use chat mode in the Docker Desktop Dashboard when you select the model in the **Models** tab. -### Push a model +### Upload a model to Docker Hub + +Use the command below to push your model to Docker Hub: + +```console +$ docker model push / +``` + +### View the logs + +Fetch logs from the Docker Model Runner to monitor activity, debug issues, or inspect inference behavior. ```console -docker model push / +$ docker model logs ``` +The following flags are accepted: + +- `-f`/`--follow`: View logs with real-time streaming +- `--no-engines`: Exclude inference engine logs from the output + ### Remove a model Removes a downloaded model from your system. From e015e276b352fa1b22de70b804d246c1adbdf3ac Mon Sep 17 00:00:00 2001 From: aevesdocker Date: Tue, 22 Apr 2025 13:25:13 +0100 Subject: [PATCH 3/8] logs command --- content/manuals/desktop/features/model-runner.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/manuals/desktop/features/model-runner.md b/content/manuals/desktop/features/model-runner.md index a8934645234..d18fa11add4 100644 --- a/content/manuals/desktop/features/model-runner.md +++ b/content/manuals/desktop/features/model-runner.md @@ -142,7 +142,7 @@ Chat session ended. ### Upload a model to Docker Hub -Use the command below to push your model to Docker Hub: +Use the following command to push your model to Docker Hub: ```console $ docker model push / @@ -150,7 +150,7 @@ $ docker model push / ### View the logs -Fetch logs from the Docker Model Runner to monitor activity, debug issues, or inspect inference behavior. +Fetch logs from Docker Model Runner to monitor activity or debug issues. ```console $ docker model logs From 799dba6a1a0e8d61d30df86b0056a76a4444f9ac Mon Sep 17 00:00:00 2001 From: aevesdocker Date: Tue, 22 Apr 2025 13:56:56 +0100 Subject: [PATCH 4/8] remove some known issues --- content/manuals/desktop/features/model-runner.md | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/content/manuals/desktop/features/model-runner.md b/content/manuals/desktop/features/model-runner.md index d18fa11add4..c91404d094c 100644 --- a/content/manuals/desktop/features/model-runner.md +++ b/content/manuals/desktop/features/model-runner.md @@ -338,20 +338,10 @@ Once linked, re-run the command. Currently, Docker Model Runner doesn't include safeguards to prevent you from launching models that exceed their system’s available resources. Attempting to run a model that is too large for the host machine may result in severe slowdowns or render the system temporarily unusable. This issue is particularly common when running LLMs models without sufficient GPU memory or system RAM. -### `model run` drops into chat even if pull fails - -If a model image fails to pull successfully, for example due to network issues or lack of disk space, the `docker model run` command will still drop you into the chat interface, even though the model isn’t actually available. This can lead to confusion, as the chat will not function correctly without a running model. - -You can manually retry the `docker model pull` command to ensure the image is available before running it again. - ### No consistent digest support in Model CLI The Docker Model CLI currently lacks consistent support for specifying models by image digest. As a temporary workaround, you should refer to models by name instead of digest. -### Misleading pull progress after failed initial attempt - -In some cases, if an initial `docker model pull` fails partway through, a subsequent successful pull may misleadingly report “0 bytes” downloaded even though data is being fetched in the background. This can give the impression that nothing is happening, when in fact the model is being retrieved. Despite the incorrect progress output, the pull typically completes as expected. - ## Share feedback Thanks for trying out Docker Model Runner. Give feedback or report any bugs you may find through the **Give feedback** link next to the **Enable Docker Model Runner** setting. From 5a55639d9e126e58891f886ca4faa6f94f267bd6 Mon Sep 17 00:00:00 2001 From: aevesdocker Date: Wed, 23 Apr 2025 10:31:07 +0100 Subject: [PATCH 5/8] add tag command --- content/manuals/desktop/features/model-runner.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/content/manuals/desktop/features/model-runner.md b/content/manuals/desktop/features/model-runner.md index c91404d094c..06628084fe0 100644 --- a/content/manuals/desktop/features/model-runner.md +++ b/content/manuals/desktop/features/model-runner.md @@ -148,6 +148,16 @@ Use the following command to push your model to Docker Hub: $ docker model push / ``` +### Tag a model + +You can specify a particular version or variant of the model: + +```console +$ docker model tag +``` + +If no tag is provided, Docker defaults to `latest`. + ### View the logs Fetch logs from Docker Model Runner to monitor activity or debug issues. From 5c35419ba9863a09217a1b520693c81c921f0183 Mon Sep 17 00:00:00 2001 From: aevesdocker Date: Wed, 23 Apr 2025 14:42:15 +0100 Subject: [PATCH 6/8] TC support --- content/manuals/desktop/features/model-runner.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/content/manuals/desktop/features/model-runner.md b/content/manuals/desktop/features/model-runner.md index 06628084fe0..3381bc3e4ca 100644 --- a/content/manuals/desktop/features/model-runner.md +++ b/content/manuals/desktop/features/model-runner.md @@ -22,6 +22,10 @@ The Docker Model Runner plugin lets you: Models are pulled from Docker Hub the first time they're used and stored locally. They're loaded into memory only at runtime when a request is made, and unloaded when not in use to optimize resources. Since models can be large, the initial pull may take some time — but after that, they're cached locally for faster access. You can interact with the model using [OpenAI-compatible APIs](#what-api-endpoints-are-available). +> [!TIP] +> +> Using Testcontainers? Testcontainers for Java now [supports Docker Model Runner](https://java.testcontainers.org/modules/docker_model_runner/) + ## Enable Docker Model Runner 1. Navigate to the **Features in development** tab in settings. From 4e04f76d7b58a4e97d80c29f4fc986fadeff6931 Mon Sep 17 00:00:00 2001 From: aevesdocker Date: Thu, 24 Apr 2025 11:28:11 +0100 Subject: [PATCH 7/8] TC support --- content/manuals/desktop/features/model-runner.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/manuals/desktop/features/model-runner.md b/content/manuals/desktop/features/model-runner.md index 3381bc3e4ca..048d93e823c 100644 --- a/content/manuals/desktop/features/model-runner.md +++ b/content/manuals/desktop/features/model-runner.md @@ -24,7 +24,7 @@ Models are pulled from Docker Hub the first time they're used and stored locally > [!TIP] > -> Using Testcontainers? Testcontainers for Java now [supports Docker Model Runner](https://java.testcontainers.org/modules/docker_model_runner/) +> Using Testcontainers? [Testcontainers for Java](https://java.testcontainers.org/modules/docker_model_runner/) and [Go](https://golang.testcontainers.org/modules/dockermodelrunner/) now support Docker Model Runner ## Enable Docker Model Runner From 6ff7dd65bd247f3fff7f4adbad3982ba66741b5e Mon Sep 17 00:00:00 2001 From: Allie Sadler <102604716+aevesdocker@users.noreply.github.com> Date: Thu, 24 Apr 2025 14:48:34 +0100 Subject: [PATCH 8/8] Update content/manuals/desktop/features/model-runner.md Co-authored-by: Sarah Sanders --- content/manuals/desktop/features/model-runner.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/manuals/desktop/features/model-runner.md b/content/manuals/desktop/features/model-runner.md index 048d93e823c..267f42cdb8b 100644 --- a/content/manuals/desktop/features/model-runner.md +++ b/content/manuals/desktop/features/model-runner.md @@ -24,7 +24,7 @@ Models are pulled from Docker Hub the first time they're used and stored locally > [!TIP] > -> Using Testcontainers? [Testcontainers for Java](https://java.testcontainers.org/modules/docker_model_runner/) and [Go](https://golang.testcontainers.org/modules/dockermodelrunner/) now support Docker Model Runner +> Using Testcontainers? [Testcontainers for Java](https://java.testcontainers.org/modules/docker_model_runner/) and [Go](https://golang.testcontainers.org/modules/dockermodelrunner/) now support Docker Model Runner. ## Enable Docker Model Runner