Skip to content

Commit 375d536

Browse files
author
福晋
committed
chore: context opt
1 parent 35b880d commit 375d536

7 files changed

Lines changed: 68 additions & 298 deletions

File tree

README.md

Lines changed: 6 additions & 82 deletions
Original file line numberDiff line numberDiff line change
@@ -8,18 +8,10 @@ A local context retrieval library that enables semantic search over your documen
88

99
## Features
1010

11-
- 📄 **Multi-format Support**: Supports Markdown, JSON, Text, and other file formats
12-
- 📚 **Multi-library Support**: Manage documents by library
13-
-**Auto-indexing**: Automatic vectorization on load with batch embedding for performance
14-
- 🔍 **Hybrid Retrieval**: Combines vector similarity + FTS text matching via RRF fusion for better recall
15-
- 🔄 **Deduplication**: Automatically skip already-loaded documents; content-hash change detection for re-embedding updated files
16-
- ⚖️ **Weight Configuration**: Per-field FTS boost weights and RRF rank constant tuning
17-
- 🛡️ **Clear Error Messages**: Throws descriptive errors when Transformers model is unavailable, guiding users to fix the issue
18-
- 🔁 **Two-stage Reranking**: KeywordReranker boosts candidates with exact query term matches after coarse vector/hybrid search
19-
- 🔁 **Two-stage Reranking**: KeywordReranker boosts candidates with exact query term matches after coarse vector/hybrid search
20-
- 🌐 **Query Expansion**: SynonymExpander uses user-provided synonym maps to bridge CN↔EN terminology gaps
21-
- 📊 **Progress Callback**: `onProgress` hook for monitoring load phases (load → embed → insert)
22-
- 🏗️ **fromDir() Quick-start**: One-call setup from a project directory with auto-derived defaults
11+
- 📄 **Multi-format Support**: Markdown, JSON, Text 文档自动加载与向量化
12+
- 🔍 **Hybrid Retrieval**: 向量语义 + FTS 全文检索双路召回,RRF 融合排序
13+
- 🔁 **Two-stage Reranking**: KeywordReranker 精排,关键词命中优先
14+
- 🌐 **Query Expansion**: 用户自定义同义词表,CN↔EN 跨语言召回增强
2315

2416

2517
## Quick Start
@@ -34,23 +26,14 @@ import { Context } from '@antv/context';
3426
// Standard creation — specify vectorsDir
3527
const ctx = await Context.create({ vectorsDir: './vectors' });
3628

37-
// Quick-start from a project directory (auto-derives basePath & vectorsDir)
38-
const ctx2 = await Context.fromDir('/path/to/project');
39-
4029
// Load documents into a specific library with automatic vectorization
4130
await ctx.load('g2', './g2-docs/**/*.md');
4231
await ctx.load('f2', './f2-docs/**/*.json');
4332

44-
// Query a single library (default: hybrid search + reranking)
33+
// Query a library (default: hybrid search + reranking)
4534
const results = await ctx.query('How to configure a line chart', { library: 'g2', topK: 5 });
4635
// => [{ content: '...', score: 0.92, scoreMode: 'reranked', id: 'g2-docs/line.md' }, ...]
4736

48-
// Query multiple libraries (array form)
49-
const crossResults = await ctx.query('chart configuration', { library: ['g2', 'f2'], topK: 5 });
50-
51-
// Query all loaded libraries
52-
const allResults = await ctx.query('visualization', { library: '*', topK: 10 });
53-
5437
// Close when done (releases resources)
5538
await ctx.close();
5639
```
@@ -138,18 +121,12 @@ Two-stage retrieval: coarse search (vector / hybrid) → reranking → final top
138121

139122
| Parameter | Type | Default | Description |
140123
|-----------|------|---------|-------------|
141-
| `library` | `string | string[]` || Library name(s). Single: `'g2'`, Multiple: `['g2', 'f2']`, All: `'*'`. Comma-separated `'g2,f2'` also supported. |
124+
| `library` | `string` || Library name to query. |
142125
| `topK` | `number` | `5` | Number of results to return |
143126

144127
```typescript
145128
// Semantic search — hybrid (vector + FTS) + reranking by default
146129
const results = await ctx.query('sankey diagram', { library: 'g2', topK: 5 });
147-
148-
// Multiple libraries
149-
const results = await ctx.query('chart', { library: ['g2', 'f2'], topK: 5 });
150-
151-
// All libraries
152-
const results = await ctx.query('chart', { library: '*', topK: 5 });
153130
```
154131

155132
#### Query Result Fields
@@ -164,60 +141,7 @@ Each result includes:
164141
| `scoreMode` | `'vector' | 'hybrid' | 'reranked'` | How the score was computed |
165142
| `meta` | `Record<string, unknown>` | Front-matter metadata (if present) |
166143
| `sourceFilePath` | `string` | Original file path relative to `basePath` |
167-
| `library` | `string` | Which library this result came from |
168-
169-
### `ctx.untrack(library, id)`
170-
171-
Remove a document from a library's dedup registry. **Important**: zvec does not support single-document deletion, so vector data remains in the store. `untrack()` only removes the dedup entry — the actual vectors remain until you call `rebuild()`.
172-
173-
| Parameter | Type | Description |
174-
|-----------|------|---------|
175-
| `library` | `string` | Library name |
176-
| `id` | `string` | Document ID to untrack from dedup tracking |
177-
178-
```typescript
179-
await ctx.untrack('g2', 'abc123__getting_started');
180-
```
181-
182-
### `ctx.rebuild(library, pattern)`
183-
184-
Rebuild a library's vector store from scratch. Deletes the existing `.zvec` store file, clears the dedup registry, and re-embeds all matching documents. Use this after `untrack()` to actually remove vectors.
185-
186-
| Parameter | Type | Description |
187-
|-----------|------|---------|
188-
| `library` | `string` | Library name to rebuild |
189-
| `pattern` | `string | string[]` | Glob pattern(s) for re-loading documents |
190-
191-
```typescript
192-
// Rebuild after untracking documents
193-
await ctx.untrack('g2', 'abc123__getting_started');
194-
await ctx.rebuild('g2', './g2-docs/**/*.md');
195-
```
196-
197-
### `Context.fromDir(dir, options?)`
198-
199-
Quick-start convenience method — creates a Context from a project directory with auto-derived defaults (`basePath` = dir, `vectorsDir` = dir/.context/vectors).
200-
201-
| Parameter | Type | Description |
202-
|-----------|------|---------|
203-
| `dir` | `string` | Project directory path |
204-
| `options` | `Partial<ContextOptions>` | Optional overrides for auto-derived defaults |
205144

206-
```typescript
207-
const ctx = await Context.fromDir('/path/to/project');
208-
// With custom overrides
209-
const ctx = await Context.fromDir('/path/to/project', { ftsFieldWeights: { content: 2 } });
210-
```
211-
212-
### `ctx.remove(library, id)`**Deprecated**
213-
214-
> Use `untrack()` instead. This alias only removes the dedup tracking entry — vector data remains in the store.
215-
> To physically remove data, call `untrack()` then `rebuild()`.
216-
217-
```typescript
218-
// Deprecated — use untrack() + rebuild() instead
219-
await ctx.remove('g2', 'abc123__getting_started');
220-
```
221145

222146
### `ctx.close()`
223147

0 commit comments

Comments
 (0)