Add LLM client, prompt builders, and prompt templates

[qu4rkn3t]

Type of change

• Refactor
• New feature
• Bug fix
• Optimization
• Documentation Update

Description

Gemini LLM client built with google-genai SDK with support for structured output, token counting, and embeddings. Also includes the flash and frontier prompt builders, which handle context-window splitting when the commit set is too large for a single prompt. The design follows spec -> provider -> model.

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.

Testing

Passed tests in Phase 2 tests

[Copilot]

Pull request overview

This PR introduces the initial building blocks for Phase 2 “commit triage” LLM support: prompt templates for the flash/frontier stages, prompt assembly/splitting logic based on token budgets, and a Google (Gemini) LLM client abstraction intended to support token counting, structured output, and embeddings.

Changes:

• Added markdown prompt templates for the “flash” and “frontier” LLM stages.
• Implemented prompt construction utilities that assemble PR/commit/diff context and split prompts to fit within a token budget.
• Added a Google GenAI (Gemini) LLM client abstraction for counting tokens, generating text/structured outputs, and producing embeddings.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
perf_keeper/commit_triage/templates/frontier.md	Adds the frontier-stage prompt template and JSON response shape guidance.
perf_keeper/commit_triage/templates/flash.md	Adds the flash-stage prompt template and JSON response shape guidance.
perf_keeper/commit_triage/prompts.py	Implements regression header + PR/commit blocks and token-budget prompt splitting.
perf_keeper/commit_triage/llm.py	Introduces the LLM client abstraction and a Google GenAI-backed implementation.

Suppressed comments (5)

perf_keeper/commit_triage/prompts.py:19

• perf_keeper.commit_triage.models does not exist in the repo; the models referenced here (CommitModel/FileModel/PRModel/RegressionContext) already exist in perf_keeper/models.py. As written, this import will raise ModuleNotFoundError.

from perf_keeper.commit_triage.models import (
    CommitModel,
    FileModel,
    PRModel,
    RegressionContext,
)

perf_keeper/commit_triage/llm.py:27

• perf_keeper.commit_triage.models is not present in the repo, and the imported symbols here (CommitRanking, FlashDecision, FlashResponse, FrontierResponse) are not defined anywhere else. This will raise ModuleNotFoundError (and also makes the __all__ entries invalid). Either add the missing models module or remove these imports/re-exports until the schemas exist.

from perf_keeper.commit_triage.models import (
    CommitRanking,
    FlashDecision,
    FlashResponse,
    FrontierResponse,
)

perf_keeper/commit_triage/llm.py:169

• complete() calls generate_content() directly, so the retry/backoff logic in _request() is bypassed. This can make prompt generation flaky under rate limits or transient 5xx errors.

        response = await self._client.aio.models.generate_content(
            model=self.model.model_id,
            contents=prompt,
            config=types.GenerateContentConfig(
                temperature=self._temperature,

perf_keeper/commit_triage/llm.py:183

• complete_structured() bypasses _request() and calls generate_content() directly, so structured generations won't be retried on 429/5xx. Using _request() here helps avoid intermittent empty/partial results under load.

        response = await self._client.aio.models.generate_content(
            model=self.model.model_id,
            contents=prompt,
            config=types.GenerateContentConfig(
                temperature=self._temperature,

perf_keeper/commit_triage/llm.py:200

• embed() bypasses _request() and calls the embedding endpoint directly, so transient rate limits / server errors won't be retried.

        logger.debug("Embedding with %s", EMBEDDING_MODEL)
        response = await self._client.aio.models.embed_content(
            model=EMBEDDING_MODEL,
            contents=text,
        )

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.