Skip to content

feat(pricing): add custom pricing overrides#549

Open
ben-vargas wants to merge 1 commit into
junhoyeo:mainfrom
ben-vargas:feat-pricing-override
Open

feat(pricing): add custom pricing overrides#549
ben-vargas wants to merge 1 commit into
junhoyeo:mainfrom
ben-vargas:feat-pricing-override

Conversation

@ben-vargas
Copy link
Copy Markdown

@ben-vargas ben-vargas commented May 13, 2026

Summary

Adds a local custom-pricing.json override layer for exact model pricing fixes. Overrides are loaded from tokscale's config directory, checked before LiteLLM/OpenRouter/Cursor fallback pricing, and use exact matching only so user-specified accounting fixes cannot fuzzy-match into unrelated models.

This is one possible implementation for #548. In the age of agentic development, maintainers may reasonably choose to have their own agents design or implement a different solution if the issue is accepted; this PR is intended as a concrete, tested example of how the feature could work.

What changed

  • Add crates/tokscale-core/src/pricing/custom.rs for loading and validating custom-pricing.json.
  • Support provider-style per-million fields such as input_cost_per_million_tokens, plus LiteLLM-compatible per-token fields such as input_cost_per_token.
  • Check custom overrides with raw exact match first, then existing synthetic /models/ normalization, before falling through to the current pricing resolver.
  • Preserve explicit source forcing: --provider litellm and --provider openrouter bypass custom overrides, while --provider custom checks only custom overrides.
  • Add tokscale pricing list-overrides with text and JSON output.
  • Document custom pricing overrides in the README.

AI disclosure

This PR was AI generated with GPT-5.5 (xhigh) and reviewed iteratively with Opus 4.7 (Max). I manually tested the resulting worktree and pushed the branch from my fork.

Validation

cargo test --workspace
cargo clippy --workspace --all-targets -- -D warnings
cargo build --release --bin tokscale

Also smoke-tested a Fireworks Kimi K2.6 Turbo override:

./target/release/tokscale pricing accounts/fireworks/routers/kimi-k2p6-turbo --provider custom --no-spinner

which reports the custom $2.00 / $8.00 / $0.30 per-1M pricing.


Summary by cubic

Add local custom pricing overrides via custom-pricing.json so users can fix or pin model prices. Overrides take precedence over LiteLLM/OpenRouter in lookups and cost calculations, unless a provider is explicitly forced.

  • New Features
    • Load exact-match overrides from ~/.config/tokscale/custom-pricing.json (or TOKSCALE_CONFIG_DIR).
    • Supports per-million fields (e.g., input_cost_per_million_tokens) and LiteLLM-style per-token fields (e.g., input_cost_per_token).
    • Lookup order: raw exact match → normalized /models/ match → existing sources (LiteLLM, OpenRouter, Cursor) → fuzzy.
    • Provider forcing preserved: --provider custom checks only custom, --provider litellm|openrouter bypasses custom.
    • Add tokscale pricing list-overrides (text or --json) to inspect loaded overrides.
    • Overrides are loaded once at startup; restart tokscale after editing the file.
    • Docs updated with schema, examples, and usage.

Written for commit 1417630. Summary will update on new commits.

@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented May 13, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
tokscale Ignored Ignored Preview May 13, 2026 3:04pm

Request Review

@ben-vargas ben-vargas marked this pull request as ready for review May 13, 2026 15:06
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 4 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="README.md">

<violation number="1" location="README.md:369">
P2: README lookup strategy should document that provider forcing changes custom override precedence</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread README.md
5. **Provider Prefix Matching** - Tries common prefixes (`anthropic/`, `openai/`, etc.)
6. **Cursor Model Pricing** - Hardcoded pricing for models not yet in LiteLLM/OpenRouter (e.g., `gpt-5.3-codex`)
7. **Fuzzy Matching** - Word-boundary matching for partial model names
1. **Custom Pricing Overrides** - Exact user-defined entries from `~/.config/tokscale/custom-pricing.json`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: README lookup strategy should document that provider forcing changes custom override precedence

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At README.md, line 369:

<comment>README lookup strategy should document that provider forcing changes custom override precedence</comment>

<file context>
@@ -357,19 +357,55 @@ tokscale pricing "grok-code"
-5. **Provider Prefix Matching** - Tries common prefixes (`anthropic/`, `openai/`, etc.)
-6. **Cursor Model Pricing** - Hardcoded pricing for models not yet in LiteLLM/OpenRouter (e.g., `gpt-5.3-codex`)
-7. **Fuzzy Matching** - Word-boundary matching for partial model names
+1. **Custom Pricing Overrides** - Exact user-defined entries from `~/.config/tokscale/custom-pricing.json`
+2. **Exact Match** - Direct lookup in LiteLLM/OpenRouter databases
+3. **Alias Resolution** - Resolves friendly names (e.g., `big-pickle` → `glm-4.7`)
</file context>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant