Skip to content

Commit 04ac002

Browse files
committed
fix(knowledge): make the conversation content hash collision-resistant
Greptile P1. The hash was `memory:{id}:{updatedAt}`, and updatedAt only has millisecond resolution. Appends landing in the same millisecond as the value already indexed hashed identically, so classifyExternalDoc called the transcript unchanged and the new messages stayed out of the knowledge base until some later write moved the clock. messageCount closes the reported path — an append always increments it — and approxBytes covers a same-count replacement. Both were already selected for the listing, so neither costs an extra query. Mutation-checked: reverting to the timestamp-only hash fails three tests. Also switches the sim auth arm's line comment to TSDoc per the repo rule (Greptile P2).
1 parent 765adba commit 04ac002

3 files changed

Lines changed: 37 additions & 5 deletions

File tree

apps/sim/app/workspace/[workspaceId]/knowledge/[id]/components/add-connector-modal/add-connector-modal.tsx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -246,7 +246,7 @@ export function AddConnectorModal({
246246
const renderAuthField = (auth: ConnectorAuthConfig) => {
247247
switch (auth.mode) {
248248
case 'sim':
249-
// Nothing to collect: the connector reads this workspace directly.
249+
/** Nothing to collect: the connector reads this workspace directly. */
250250
return null
251251

252252
case 'apiKey':

apps/sim/connectors/sim-conversations/sim-conversations.test.ts

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -212,7 +212,7 @@ describe('conversationToStub', () => {
212212

213213
it('hashes on the update watermark, which moves whenever a message is appended', () => {
214214
const base = conversationToStub(BASE_ROW).contentHash
215-
expect(base).toBe('memory:mem-1:2026-01-02T00:00:00.000Z')
215+
expect(base).toBe('memory:mem-1:2026-01-02T00:00:00.000Z:6:1024')
216216

217217
const appended = conversationToStub({
218218
...BASE_ROW,
@@ -227,6 +227,30 @@ describe('conversationToStub', () => {
227227
conversationToStub({ ...BASE_ROW }).contentHash
228228
)
229229
})
230+
231+
/**
232+
* `updatedAt` is only millisecond-resolution, so appends landing in the same
233+
* millisecond as the indexed value would otherwise hash identically and the sync
234+
* engine would call the transcript unchanged — leaving the new messages out of
235+
* the knowledge base until some later write moved the clock.
236+
*/
237+
it('distinguishes appends that share a millisecond with the indexed value', () => {
238+
const indexed = conversationToStub(BASE_ROW).contentHash
239+
const appendedSameMs = conversationToStub({
240+
...BASE_ROW,
241+
messageCount: BASE_ROW.messageCount + 2,
242+
approxBytes: BASE_ROW.approxBytes + 180,
243+
}).contentHash
244+
245+
expect(appendedSameMs).not.toBe(indexed)
246+
})
247+
248+
/** A same-count replacement still moves the byte size. */
249+
it('distinguishes a same-millisecond, same-count content replacement', () => {
250+
expect(
251+
conversationToStub({ ...BASE_ROW, approxBytes: BASE_ROW.approxBytes + 40 }).contentHash
252+
).not.toBe(conversationToStub(BASE_ROW).contentHash)
253+
})
230254
})
231255

232256
describe('cursor', () => {

apps/sim/connectors/sim-conversations/sim-conversations.ts

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -164,8 +164,16 @@ export function renderTranscript(
164164
* Builds the listing stub for one conversation.
165165
*
166166
* Single source of truth for `contentHash`, used by both listing and hydration.
167-
* `updatedAt` is the right watermark here: `Memory.appendMessage` bumps it on every
168-
* appended message, so it moves exactly when the transcript changes.
167+
*
168+
* `updatedAt` is the primary watermark — `Memory.appendMessage` bumps it on every
169+
* appended message — but it only has millisecond resolution, so two appends landing
170+
* in the same millisecond as the previously indexed value would hash identically and
171+
* `classifyExternalDoc` would call the transcript unchanged, leaving the new messages
172+
* out of the knowledge base until some later write moved the clock.
173+
*
174+
* `messageCount` closes that: an append always increments it. `approxBytes` covers
175+
* the rarer case of a same-count replacement. Both are already selected for the
176+
* listing, so neither costs an extra query.
169177
*/
170178
export function conversationToStub(row: ConversationRow): ExternalDocument {
171179
return {
@@ -175,7 +183,7 @@ export function conversationToStub(row: ConversationRow): ExternalDocument {
175183
contentDeferred: true,
176184
mimeType: 'text/plain',
177185
// No sourceUrl: conversations have no page of their own in the app.
178-
contentHash: `memory:${row.id}:${row.updatedAt.toISOString()}`,
186+
contentHash: `memory:${row.id}:${row.updatedAt.toISOString()}:${row.messageCount}:${row.approxBytes}`,
179187
metadata: {
180188
conversationId: row.key,
181189
messageCount: row.messageCount,

0 commit comments

Comments
 (0)