Skip to content

flm: pull cause long model load time #3129

Description

@Rimits

Platform

Windows

Lemonade Version

11.5.2

GPU / APU Model

hx 370

Component

lemond (server)

Bug Description

Every time flm loads a model, lemonade forces a pull before serveing the model, even after the model is already downloaded.
but flm is for npu, for laptops really, but nowadays many laptops only have wifi which means not always reliable connections, the pull could stuck for minutes, always pull before serving makes no sense.

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingengine::flmFastFlowLM backend (NPU); multi-modal LLM/ASR/embeddings/reranking

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions