Add Pi.dev AI assistant: per-device chat + scheduled AI tasks - #2509
Add Pi.dev AI assistant: per-device chat + scheduled AI tasks#2509teknoprep wants to merge 85 commits into
Conversation
012fd98 to
ccb8396
Compare
Adds an in-portal AI assistant scoped to a single device, scheduled AI tasks, and bulk AI commands across many devices. All device actions go through the existing TRMM REST API. Backend: - AIProvider / AIModel catalog and CoreSettings toggles - Role can_use_ai / can_use_ai_autoapprove / ai_allowed_models - PiPerms / AITaskPerms / BulkAIPerms; short-lived redis session tokens - /agents/<id>/pi/ session + history endpoints - /core/ai/ providers, models, available-models, tasks, runs, bulk endpoints - AITask / AITaskRun with Celery poller + runner; verdict -> custom alert - task + bulk scheduling: run Now (one-shot, self-disabling) or Scheduled (interval/daily/weekly/monthly); shared next-run computer - BulkAICommand: target All/Client/Site/Agents or dynamic Filter rules, skip offline agents; bulk target preview returns matched online devices - PI_RUN_TIMEOUT (default 3600s) with redis verdict recovery Bridge (pibridge/): - Node service embedding the assistant runtime, deployed to /opt/pi-trmm-bridge; device-scoped tools call the TRMM REST API only - Linux command timeout guard, per-session logging, WebSocket heartbeat Install/update: - install.sh adds the nginx /pi/ location and calls pibridge/setup.sh - update.sh calls pibridge/setup.sh (idempotent) See PI_AI_ASSISTANT.md for full documentation.
ccb8396 to
4a45ccb
Compare
Multi-machine mode: - POST /agents/pi/multisession/ creates a session spanning 2-8 agents, each with an operator-written role note (e.g. 'primary Proxmox node', 'PBS'); per-agent permission enforcement, per-machine audit entries - bridge tools gain a required 'machine' param in multi mode (hostnames deduped #N); model cannot target anything outside the session's set - multi system prompt: per-machine facts + role notes + coordination rules; approval prompts prefixed with the target hostname; mixed OS sets supported Email: - POST /core/ai/email/ sends plain-text mail via the existing alerting SMTP config (CoreSettings.send_mail override_recipients); recipient validation, Debug Log audit; uses send_mail test-mode so real SMTP errors are returned - new send_email bridge tool: approval-gated in chat, auto-approved in unattended task runs, kept in read-only mode (not a device mutation) Reliability: - every bridge->TRMM call bounded by a transport timeout (120s default, cmd timeout+30s for command/script runs) and wired to the turn's abort signal; timeouts surface as STALLED: tool errors the model can react to - turn-stall watchdog only fires when no tool is in flight (long device commands no longer aborted at 180s) - install.sh: uwsgi_read_timeout 300s -> 940s so up-to-900s AI commands aren't 504'd Docs: PI_AI_ASSISTANT.md updated for all of the above.
|
Pushed Multi-machine chat (2–8 devices per conversation)
|
Multi-machine chats recorded their history only under the primary agent and didn't store the machine set, so resuming rebuilt a single-machine session. Record multi + machines (agent_id/hostname/role) in the history index so the UI can resume the full multi-machine session.
|
Follow-up |
…losed targeting Read-only role: - Role.can_use_ai_mutate (migration 0044); when off the AI session is read-only: write tools removed + run_command refuses destructive commands (bridge classifier), read-only badge in chat, RO notice in the prompt Bulk filter builder: - grouped AND/OR rule builder (filter_match + per-group match; migration 0062); legacy flat filters auto-migrate to one AND group - new 'installed software (name)' field: substring match over each machine's software inventory (find e.g. every machine with 'online backup') Safe targeting (fixes fail-open bug that fanned out to the whole fleet): - _resolve_bulk_targets now FAILS CLOSED: empty/ineffective filter, missing client/site, or unknown target -> zero agents (only explicit 'all' hits all) - hard cap PI_BULK_MAX_AGENTS (default 250): runs over the cap are refused and logged; preview shows an over-cap warning and disables Save Kill switch: - per-command Stop (/core/ai/bulk/<id>/stop/) and emergency Stop-all (/core/ai/stop-all/); bridge /pi/run/abort aborts in-flight headless runs - runner tasks re-check enabled + a redis emergency-stop flag at execution time, so a queued backlog drains as no-ops (no LLM calls, no alerts) Also: AISendEmail already present; docs (PI_AI_ASSISTANT.md) updated throughout.
|
Pushed Read-only role — Bulk filter builder — grouped AND/OR rule builder ( Fail-closed targeting (important fix) — Kill switch — per-command |
…ail From, run disabled one-shot; add missing migrations - Bulk results viewer: GET /core/ai/bulk/<id>/results/ returns the latest run per agent (with client/site to disambiguate same-named servers); computers-left / results-right UI, double-click row to edit - Exclude machines: BulkAICommand.exclude_agent_ids (migration 0063) drops specific agents from the resolved set even if they match; preview marks excluded and counts effective targets - Run now works on disabled/one-shot commands: per-agent guard now keys on a per-command stop flag + global kill flag (not enabled); Run now clears the stop flag and re-dispatches. Scheduled auto-runs still gate on enabled - Configurable email From: send_mail override_from/override_from_name; /core/ai/email builds pi-<job|random>@<smtp-domain> by default, bare word -> local part on smtp domain, full address used verbatim; bridge send_email gains from_address/from_name and passes job_ref (run id) - Client/Site added to AITaskRunSerializer - FIX: add migrations that a prior 'git commit -am' dropped (untracked): accounts 0044 (can_use_ai_mutate), core 0062 (filter_match), core 0063 Docs (PI_AI_ASSISTANT.md) updated.
|
Pushed |
- Agnostic, settings-defined helpdesk integration: natural-language policy prompt + admin 'helpdesk.js' code box (ai_helpdesk_prompt / api / code) run on the bridge; the AI calls named operations (create_ticket, reply, note, submit_report, ...). Works with any helpdesk/PSA, no vendor code shipped. - 'Use AI to Help Create These' assistant endpoint (HelpdeskAssist) that interviews the admin and drafts both the policy and the integration code. - Auto-ticketing: correct customer (closest-match + safe fallback), dedup + escalate, reopen closed tickets, one combined end-of-batch report + individual failure tickets, device deep-link in tickets, customer-safe replies. - created_by/modified_by stamping on AI tasks & bulk commands. - Client/Site 'AI History' review across all machines (ai/history-scope). - Migrations 0064-0067.
619b164 to
bc914fc
Compare
…anage-all-AI-tasks permission, richer setup interview - Colorized HTML 'terminal cards' for command/log output in tickets (helpdesk.js toHtml; policy-guided) - Distinct 'AI Assistant Session' audit action (audit_ai_session) so AI chats aren't logged as remote-control sessions - can_manage_all_ai_tasks role permission: manage/delete AI tasks & bulk commands from any owner, incl. outside agent scope - ai_alerts_only_on_ticket_error setting - Setup assistant now interviews across full capability set (ownership resolution, HTML colorization, open/close/assign-tech/assign-end-user, templated replies, dedup, combined reports, get_ticket)
…t-builder assistant - bridge: preserve device_facts through buildTools so the ticket 'jump to device' deep link is populated for single-device runs (was dropped by the machines map) - bridge: apiErrorMessage() surfaces the human-readable provider error message (e.g. usage-limit/rate-limit notices) in run/report/chat results instead of a raw blob - AI prompt-builder assistant: interviews the admin and drafts the prompt (and, for bulk, the combined-report instruction) for AI Tasks and Bulk AI Commands (bridge task_prompt mode + AIPromptAssist view + ai/prompt-assist route)
…rompt) - Agent.ai_notes (TextField, migration 0063): distilled per-device memory - device_facts now carries ai_notes for both interactive and headless/bulk runs - AIDeviceNote view + /core/ai/device-note/ route (GET/POST-append/PUT-replace, X-API-KEY service auth for the bridge, per-agent perm check) - bridge: inject saved notes into single- and multi-device system prompts as trusted prior-run context, and add a save_device_note tool (not gated, allowed read-only) that appends via trmm.saveDeviceNote
- enforce brevity server-side: each note collapsed to ONE short line and capped at 200 chars; total memory bounded to 1500 chars/device (oldest drops first); near-duplicate notes are skipped; date-only timestamps - tighten save_device_note tool + system-prompt guidance to be terse and avoid duplicating already-saved facts
- core.send_mail gains optional html_body (backward compatible): when set, adds an HTML alternative so HTML clients render it and others fall back to plain text - AISendEmail accepts an optional 'html' body and passes it through - bridge send_email tool gains an optional 'html' param (inline-styles guidance); system prompt tells the model it can send formatted HTML with a plain-text fallback
- CoreSettings: ai_ticket_automation_enabled (kill switch), ai_ticket_scope (JSON domain/alert scope limiter), ai_ticket_triage_prompt (admin-defined triage policy) - all editable on the fly in Global Settings (migration 0069) - AITicketState model: per-ticket dedup/audit state (which tickets were seen, classification, proposed action); first poll baselines the existing backlog - celery: poll_helpdesk_tickets (beat, 90s) lists open tickets via the admin-defined helpdesk.js list_open_tickets op and scope-limits deterministically; triage_ai_ticket runs one shadow triage per new ticket - bridge: /pi/tickets/poll (thin pass-through) + /pi/ticket-triage session with ONLY read (get_ticket) + submit_triage tools; the staff-only shadow note is posted deterministically afterwards - the model cannot close/reply/act
- CoreSettings.ai_ticket_act_on_alerts (migration 0070): when on, the AI ACTS on alerts instead of only drafting - non-actionable alerts (e.g. successful backups) are moved to Cancelled via the helpdesk.js cancel_ticket op; actionable alerts are claimed (assigned to the AI bot) + annotated, left open for work - bridge triage runner: deterministic action based on the model's verdict (model still has only read + submit_triage tools; it never acts itself). New helpdesk ops used: cancel_ticket / claim_ticket (defined per-deployment in helpdesk.js) - triage task records action-based status (cancelled_clean / actionable_claimed)
…input - AIResolveDevices endpoint (/core/ai/resolve-devices): links a helpdesk company (domain/name) + requester username to the RMM client (override map -> exact -> fuzzy token match) and the user's device(s) + candidates - CoreSettings.ai_ticket_client_map (migration 0071): overrides for the company->client link when fuzzy match isn't confident - bridge triage now RESOLVES before deciding: read-only tools resolve_client / find_devices / list_kb_articles / get_kb_article; verdict gains needs_input + client + affected_device. When needs_input, code tags the ticket (helpdesk.js set_needs_input_tag) and posts the draft - never auto-acts - trmm.resolveDevices client; requester_email threaded through triage - helpdesk-agnostic: the actual ticketing/KB ops live in Global Settings helpdesk.js
…t clients - LOOK-AT (triage): look_at_all_unassigned triages every unassigned/bot-assigned ticket (any domain) - posts a look-only shadow note - AUTO-ACTION (cancel/claim/tag): gated to auto_action_domains (requester domain) OR auto_action_clients (resolved client name - covers infra alerts with no requester domain). Decided in the bridge AFTER the AI resolves the client - non-auto-action tickets are never modified (shadow note only)
- AIDecisionRequest model (migration 0072): pending human-decision + Q&A thread, keyed by a token that backs a deep link - triage now generates a decision URL and includes it in the needs_input ticket note; persists an AIDecisionRequest when it tags Johnny 5 - AIDecisionView (GET thread / POST tech reply / close) -> bridge /pi/decision - bridge runDecisionChat + buildDecisionTools: the tech answers, the AI continues ON THE TICKET (helpdesk_call: reply/note/close/cancel/clear-tag/KB + find_devices), no device shell. Stateless (thread replayed each turn)
- AIScheduledAction model (migration 0073): device + run_at + action; runs once - dispatch_due_ai_scheduled_actions (beat, 60s) = cheap DB timestamp check (no LLM); run_ai_scheduled_action executes via the device-run path and updates the ticket, then DELETES the job on success (keeps status=error on failure for review) - AIScheduleAction API (create/list/delete) + trmm.scheduleAction; schedule_action tool added to the DECISION chat only (human-directed) - NOT to triage (no auto-scheduling) - decision-chat prompt: [Alert]->cancel / non-alert->AI Closed routing rule, one-ticket scope, and scheduling guidance
… the email SIGNATURE (sender may differ / shared mailbox), pass username + name; a printer/scanner is not an RMM agent so resolve the user's workstation, not the peripheral
… Smith) - no real user/person names in shipped code
…old labels, section spacing, rule separators, styled chat link) instead of one dense run-on line; idempotent HTML escaping
…ncl. non-requesters) instead of emailing separately
… message (matched by RMM email->Odoo user; takes over unassigned/bot-owned only, never steals from a human); dedup add_follower against existing followers by id/email/similar name
…company/contact re-attribution to automation-originated tickets (alert_clean/alert_actionable) only; regular/unknown left as-is so the real requester is preserved
…) to decision + device chats, alongside save_device_note
…not a 'change' - capture proactively/silently even when told not to make changes; those constraints govern devices/customer comms, not memory
…e store (symptom/cause/fix/verify) built by mining closed tickets. Adds AIProcedure model+migration, CRUD API, scheduled miner (editable enable/interval/backfill/prompt in Global Settings), bridge miner endpoint + submit_procedures tool
…bled - only the SCHEDULED miner requires the mining toggle + interval
…ining never re-processes unchanged tickets (only new/changed-since-last-look); chunked mining so every ticket is actually read; scan the full backfill window each run
…; searchable by code
…ic procedures AND per-company Odoo KBs) with a resolution-quality gate; live mining progress in Redis + status endpoint
…code + exact + fuzzy title match); Stop button for the live miner (Redis flag, graceful stop between companies)
…iner + manual create/edit), constrain the miner to a fixed list, dedupe near-duplicate categories; return all_categories to the UI
…next batch until the whole window is done (stops on Stop or when nothing remains); bridge reports 'more'
… (add/remove user, permissions, licenses, delegates, password/MFA) require the requester to be the company Primary/Secondary Support Contact (check_support_authorization) before any auto-action, even in Write mode
…ks identity/access commands (user add/remove, group/permission changes, licenses, mailbox delegates, password/MFA) unless the ticket requester is an approved support contact; applies even in Write mode/auto-approve
…ce memory (get/save_device_note), direct on-box access (no tokens needed), built-in ticketing dedupe, and web research; stops interrogating the admin about plumbing and references real mechanisms
…y any AI reply (markdown/plain-text -> clean branded HTML); policy tells the model to write simple markdown (auto-converted) instead of hand-writing HTML, for both the decision chat and device chat
…tabular data (never space-aligned text) and fenced blocks for output; HTML or markdown both fine (beautify renders markdown tables + preserves line breaks). Reverts the markdown-only push that made the model flatten tables
…cked ticket is absent from a successful stage batch, remove its stale AITicketState/decision rows so deleted tickets no longer show in the console
…model catalog - capabilities.js (new): capability CLASSES + surface->classes table + default-deny. Unattended runs no longer hold the full ticket surface; an operation the integration declares mutating but leaves unclassified is denied. Classes (not operation names) keep this portable across helpdesks. - tools.js: buildTools/buildDecisionTools take a `surface`; helpdesk_call enforces the capability check at execute time and only advertises permitted operations. Removes the hardcoded blockOps operation-name deny-list, which failed open on any other helpdesk. close-class operations are gated and never auto-approvable. - helpdesk-runtime.js: exposes exports.opClasses from the integration. - server.js: durable audit logging of unattended tool calls (previously none) with outbound customer message bodies recorded in full; /pi/helpdesk-caps for capability provenance; /pi/models, /pi/models/register, /pi/busy, /pi/restart; single registry builder so every surface sees registered models. - models-catalog.js (new): query each provider's own API for available models. - verifier-runtime.js (new): sandboxed alert-verifier loader/matcher. No deployment-specific data: integration code, prompts, verifier rules and credentials remain in the live database only.
This was the only CRLF-terminated file in the tree, which would otherwise make the next commit look like a full-file rewrite. Separated so the functional diff is reviewable.
…model catalog - Intake: tickets this system files as internal notices for humans are classified deterministically (info / info_for_human), never sent to the model, never auto-closed, and released to unassigned. Two such notices had been auto-cancelled within a minute of creation, which for the urgent variant would have swallowed an outage warning. Which tickets qualify is decided by the integration, so product code stays helpdesk-agnostic. - Scheduling: recurring AI tasks and bulk commands advance next_run at DISPATCH instead of at completion. Beat runs every minute while a run takes minutes, so tasks were dispatched once per minute of their own runtime (measured 1.8x-9.0x per slot, duplicating side effects including customer contact). Also ends infinite re-dispatch of a run that aborts early. Trade-off: a crashed run waits for its next slot rather than retrying. - AIHelpdeskCaps: reports each helpdesk operation's capability class and the PROVENANCE of that class (declared / name-guess / unclassified), so an integration edit that leaves an operation unclassified is visible before enforcement denies it. - Scheduled AI runtime updates: nightly window, quiescence-gated (refuses while any chat, run, mining job, task or triage is in flight; an unreachable bridge counts as busy), compatibility probe with rollback, and a blocked-version guard so a failing release is not retried nightly. - Model catalog: discovery via each provider's own API, with registration of models the runtime does not yet know. - Migrations 0080-0086 for the above settings.
… is unclassified An unattended run holds no `customer` capability, so a scheduled task cannot email a customer at all. Some legitimately should. Rather than widening the surface for every unattended run, authorisation is declared per task: - AITask.reply_register = none | general | technical (default none). How technical a reply should be is the task author's judgement, not the model's, and not a global policy - so it is declared per task and travels with the run. - capabilities.GRANTABLE caps what a per-run authorisation can add at ['customer'], so a task can never grant itself closing or routing authority however it is configured. Verified: with the grant, reply_to_ticket is permitted and every close-class operation is still refused. - The register also shapes the prompt, and one rule applies at every register: never disclose credential locations, secret file paths, permissions or access mechanics. Ticket email is forwarded and archived outside either party's control, so naming where credentials live carries risk with no benefit to the reader. Also: check_ai_capability_health (hourly, deduped daily) files an internal notice ticket when an operation has no capability class. Product code denies such operations, which is correct but silent - it previously took an integration edit wiping every tag with no symptom to discover that. The deny is unchanged; only the consequence becomes visible, and internal notices are never auto-closed.
Turning enforcement on can silently break working features in two ways that were both found by accident rather than by design: an integration edit can leave an operation unclassified (enforcement then refuses it), and a task that legitimately emails customers without a declared reply register loses that ability. report_caps_enforcement_readiness runs the pre-flight checks deterministically each morning after the overnight scheduled jobs, gathers what warn mode actually observed in the last 24h, and emails a GO / NO-GO with the evidence: - every operation classified - every customer-emailing task authorised - nothing refused during warn mode - no scheduled slot dispatched more than once (duplicate-dispatch fix still holding) It reports only. The mode lives in the bridge environment and changing it needs a bridge restart, which is not something to do unattended off the back of a report.
The previous commit hardcoded an operator email address in product code. Product code must carry no customer or operator identifiers - recipients are configuration. Now reuses the daily report recipient list, falling back to the configured alert recipients, and no-ops with a clear result if neither is set.
…r, operator-defined reports
Adds the pieces that make the assistant safe to leave running, and documents the
whole feature set in FEATURES.md so the PR can be evaluated without reading the diff.
Deterministic layer (authority as data, not prose):
* capability classes per helpdesk operation, per surface, default-deny; ships in
warn mode and flips to enforce on evidence
* a condition engine that recognises recurring notifications from APPROVED
procedure rows - declarative match specs, never an expression - so vendor
knowledge is data a technician can read, edit and approve
* a recurrence ledger: advise once, suppress identical repeats against an open
tracker, stand down when the condition stops, and treat a human closing the
tracker as a decision
* instructed vs self-directed: a technician's instruction IS the approval and the
authorising sentence is written onto the ticket; the MODEL acting on its own
initiative still asks a human every time and no toggle can skip it
Work ledger (measured time, not guessed time):
* four ingest sources - ticket chat, device chat, RMM activity, hand-written
helpdesk work - each row stamped with the rule that produced it
* a human driving the AI owns that time; parallel work counts in full;
corrections are new rows so a sent report still reproduces exactly
* written work is valued by content as well as elapsed time
* staff identified by mail domain (a customer portal login is not a technician),
identity by user id so a rename cannot fork someone's history
* coverage reported as timed / seen-but-not-timed / no-activity, because work that
leaves no trace must not be presented as zero
Reports:
* AIReportSchedule: operator-defined reports at any cadence with their own window,
recipients, options and extra prompt instructions; one dispatcher replaces the
two hard-wired report jobs
* open-ticket review that reads each ticket's recent thread before judging it, so
it cannot recommend work that is already done
Also: models.json registrations now expire once the runtime learns the model
natively (a stale stub silently downgraded a model and made every call to it fail),
and provider rejections are logged and surfaced instead of looking like an empty
answer.
03824fe to
d695e0a
Compare
📖 Full feature documentation (rendered): https://github.com/teknoprep/tacticalrmm/blob/feature/pi-ai-assistant/PI_AI_ASSISTANT.md
🔗 Frontend PR: amidaware/tacticalrmm-web#56
Summary
Adds an in-portal AI assistant scoped to a single device, plus scheduled AI tasks that periodically check a device and raise alerts on findings. All device actions go through the existing TRMM REST endpoints (audited, over the normal agent path).
What's included
AIProvider,AIModel) +CoreSettingstoggles; models auto-listed from configured keys.can_use_ai,can_use_ai_autoapprove,ai_allowed_models(enforced server-side).POST /agents/<id>/pi/session/mints a short-lived redis token; a Node bridge (pibridge/, deployed to/opt/pi-trmm-bridge) runs the assistant with device-scoped tools only.AITask/AITaskRun): Celery beat poller + runner; areport_resultverdict maps to a custom TRMM alert by threshold. Read-only by default for unattended runs.Install / update
install.shadds the nginx/pi/location and callspibridge/setup.sh;update.shcalls it too (idempotent) — no manual steps.Full documentation: PI_AI_ASSISTANT.md.
Pairs with the tacticalrmm-web PR of the same name.