Skip to content

Add distill-json and semantic-llm-cache to Development/Tools#142

Open
karthyick wants to merge 1 commit into
reorx:masterfrom
karthyick:add-distill-and-cache
Open

Add distill-json and semantic-llm-cache to Development/Tools#142
karthyick wants to merge 1 commit into
reorx:masterfrom
karthyick:add-distill-and-cache

Conversation

@karthyick
Copy link
Copy Markdown

Adding two Python tools to Development → Tools that reduce OpenAI API costs:

distill-json

Lossless JSON compression for ChatGPT/OpenAI API payloads. Reduces tokens 60-85% on bulk repeated records via schema extraction, dictionary encoding, and equivalence partitioning. Drop-in replacement for json.dumps().

semantic-llm-cache

Decorator-based semantic caching for OpenAI API calls. Skip 20-40% of API calls when prompts are semantically similar. Lightweight Python alternative to GPTCache (which is already in this section).

Both:

  • pip install distill-json / pip install semantic-llm-cache
  • MIT licensed

Thanks for maintaining this list!

Two Python tools for reducing OpenAI API costs: token compression and semantic response caching.
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds two new tools, distill-json and semantic-llm-cache, to the README.md file. The review feedback highlights the need to update the Chinese version of the documentation for consistency and suggests removing the pip install commands from the tool descriptions to match the existing style of the list.

Comment thread README.md

Calculate how much it will cost to generate certain number of words by using OpenAI GPT-3.5 API.

- [distill-json](https://github.com/karthyick/DISTILL)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The README.cn.md file is currently out of sync as it does not include these new entries. To maintain consistency across the bilingual documentation, these tools should be added to the Chinese version as well.

Comment thread README.md

- [distill-json](https://github.com/karthyick/DISTILL)

Lossless JSON compression for ChatGPT/OpenAI API payloads. Reduces tokens 60-85% on bulk repeated records via schema extraction, dictionary encoding, and equivalence partitioning. Drop-in replacement for `json.dumps()` — output stays human-and-LLM-readable. `pip install distill-json`.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The inclusion of pip install commands in the description is inconsistent with other entries in this section. Most items focus on the tool's utility and leave installation details to the linked repository.

Suggested change
Lossless JSON compression for ChatGPT/OpenAI API payloads. Reduces tokens 60-85% on bulk repeated records via schema extraction, dictionary encoding, and equivalence partitioning. Drop-in replacement for `json.dumps()` — output stays human-and-LLM-readable. `pip install distill-json`.
Lossless JSON compression for ChatGPT/OpenAI API payloads. Reduces tokens 60-85% on bulk repeated records via schema extraction, dictionary encoding, and equivalence partitioning. Drop-in replacement for `json.dumps()` — output stays human-and-LLM-readable.

Comment thread README.md

- [semantic-llm-cache](https://github.com/karthyick/prompt-cache)

Decorator-based semantic caching for ChatGPT/OpenAI API calls. Wrap any LLM call with `@semantic_cache` and skip 20-40% of API calls when prompts are semantically similar. Lightweight Python alternative to GPTCache. `pip install semantic-llm-cache`.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similar to the entry above, the pip install command should be removed to maintain consistency with the rest of the list.

Suggested change
Decorator-based semantic caching for ChatGPT/OpenAI API calls. Wrap any LLM call with `@semantic_cache` and skip 20-40% of API calls when prompts are semantically similar. Lightweight Python alternative to GPTCache. `pip install semantic-llm-cache`.
Decorator-based semantic caching for ChatGPT/OpenAI API calls. Wrap any LLM call with `@semantic_cache` and skip 20-40% of API calls when prompts are semantically similar. Lightweight Python alternative to GPTCache.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant