Skip to content

feat: rebuild plugin on OS on-device models - #1

Merged
wojt-janowski merged 1 commit into
mainfrom
feat/os-on-device-models
Oct 29, 2026
Merged

wojt-janowski merged 1 commit into
mainfrom
feat/os-on-device-models

Conversation

@wojt-janowski

Copy link
Copy Markdown
Contributor

Rebuilds the plugin from scratch on top of the operating system's built-in on-device models. iOS uses Apple Foundation Models and Android uses Gemini Nano through ML Kit GenAI. The plugin no longer downloads, bundles or runs its own models. It removes the MLX/llama.cpp feasibility probe, the model manifests, the download tools and the old design docs.

What's included

  • PHP API. LocalAI::availability(), download(), generate(), chat() and generation($id). A Generation handle offers status(), text(), cancel(), result() and snapshot(). Errors are typed, and the only options are temperature and maxTokens; anything else is rejected.
  • Native runtimes (Swift and Kotlin). Each platform has one process-wide FIFO queue (up to 8 pending) and runs one generation at a time. Cancellation is cooperative. Snapshots of the last 32 results stay in memory. Queued work waits while the app is in the background.
  • Events. TextDeltaGenerated, GenerationCompleted, GenerationFailed, GenerationCancelled and ModelDownloadProgress. All implement DispatchableFromNative, which NativePHP 4.6 requires before it delivers an event to PHP.
  • Android Kotlin hook. Every ML Kit GenAI Prompt API release ships Kotlin 2.2–2.4 metadata, which NativePHP 4.6.1's Kotlin 2.0.0 template cannot read. A pre_compile hook raises the host to Kotlin 2.3.21 and migrates kotlinOptions to compilerOptions. It never lowers a newer version, and should do nothing once Build the Android template on current and future JDKs NativePHP/mobile-air#552 is released.
  • MIT licence.

Verification

  • vendor/bin/pest: 15 tests pass, including the hook's edits against NativePHP's real Gradle template.
  • Android: on a fresh NativePHP 4.6.1 host, running the plugin compiler (which runs the hook) and then assembleDebug succeeded. The APK contains the plugin and ML Kit classes, and the bridge functions are registered.
  • iOS: not compiled. No Mac was available.
  • Neither platform has been tested on a physical device. On iOS 18–25 devices, the app launching depends on Foundation Models being weak-linked, which still needs checking.

Amp thread: https://ampcode.com/threads/T-01a1866d-8add-73f3-a00b-1cc5a82f4ef7

Replace the MLX/llama.cpp feasibility probe with a plugin that uses only
the operating system's built-in model: Apple Foundation Models on iOS and
Gemini Nano through ML Kit GenAI on Android. Models are no longer
downloaded or bundled by the plugin.

- PHP API: availability, download, generate, chat, generation handles,
  cancel and snapshots, with typed errors and strict option validation
- Native runtimes: process-scoped FIFO queue, one active generation,
  cooperative cancellation, in-memory snapshots, background pause
- Events implement DispatchableFromNative for NativePHP 4.6
- pre_compile hook raises the Android host to Kotlin 2.3.21 and migrates
  kotlinOptions to compilerOptions, because ML Kit GenAI ships Kotlin 2.4
  metadata that NativePHP's Kotlin 2.0 template cannot read
- MIT licence

Amp-Thread-ID: https://ampcode.com/threads/T-01a1866d-8add-73f3-a00b-1cc5a82f4ef7
Co-authored-by: Amp <amp@ampcode.com>
@wojt-janowski
wojt-janowski merged commit dcda6c9 into main Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant