Benchmark for Chat History Compression Algorithms in Agentic Systems with Function Calling#13

Open

mikhail-kharlamov wants to merge 37 commits intoemnigma:mainfrom

mikhail-kharlamov:benchmarking-first-stage

Commits on Nov 9, 2025

feat: added launch of new evaluation
michaelharlam
committed
refactoring 1
michaelharlam
committed
refactoring 2
michaelharlam
committed
refactoring 3
michaelharlam
committed
feat: evaluating is ready for launching
michaelharlam
committed

Commits on Nov 12, 2025

feat: added f1 score tool evaluator
michaelharlam
committed
feat: launch f1
michaelharlam
committed

Commits on Nov 15, 2025

feat: added statistics
michaelharlam
committed

Commits on Nov 16, 2025

feat: added working statistics calculating
michaelharlam
committed
feat: jsonl -> json in memory
michaelharlam
committed

Commits on Nov 17, 2025

feat: refactoring 1
michaelharlam
committed
feat: added parsing new data type
michaelharlam
committed

Commits on Nov 18, 2025

feat: added color logs, shuffled launch
michaelharlam
committed

Commits on Dec 25, 2025

benchmark + new algorithms
michaelharlam
committed

Commits on Dec 27, 2025

feat: rewrite from string prompt to list of messages (only baseline dialogue)
michaelharlam
committed

Commits on Dec 28, 2025

feat: added DialogueWithWeights
michaelharlam
committed

Commits on Jan 6, 2026

fix: style
michaelharlam
committed

Commits on Jan 7, 2026

feat: added langchain messages to memory algs. also changes by mypy, by ruff and in tests
michaelharlam
committed
fix: ruff and mypy
michaelharlam
committed
fix: some important constants move to .env
michaelharlam
committed
fix: mypy & ruff
michaelharlam
committed
fix: pytest version
michaelharlam
committed

Commits on Jan 24, 2026

feat: added separation into a main model and the model for memory; added feature for using local models
michaelharlam
committed
fix: deleted extra file
michaelharlam
committed
feat: added feature for using local models in the dialogue_baseline
michaelharlam
committed

Commits on Jan 28, 2026

fix: memory storage
michaelharlam
committed
fix: by ruff
michaelharlam
committed

Commits on Jan 29, 2026

fix: removed comments & hardcode
michaelharlam
committed
feat: added docs to the main classes and methods
michaelharlam
committed

Commits on Jan 31, 2026

feat: project restructured, added makefile (now it doesn't work), prompt is divided, all of system messages are assembled into the one
michaelharlam
committed
fix: refactored
michaelharlam
committed

Commits on Feb 1, 2026

fix: removed vestige prompt
michaelharlam
committed
fix: removed vestige prompt
michaelharlam
committed

Commits on Feb 4, 2026

feat: refactoring, added more tests
michaelharlam
committed

Commits on Feb 10, 2026

feat: added semantic similarity for f1
michaelharlam
committed

Commits on Feb 11, 2026

feat: added calculating by logs, args similarities, the stub od the llm-as-a-judge
michaelharlam
committed
feat: llm as a judge template is updated
michaelharlam
committed