Skip to content

fix(opencode): support SQLite database for OpenCode >= 1.2.2 #850

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dobbymaniac wants to merge 1 commit into
base: main
Choose a base branch
from
+202 −69
Open

fix(opencode): support SQLite database for OpenCode >= 1.2.2 #850

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions apps/opencode/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,5 +78,8 @@
"onFail": "download"
}
]
},
"dependencies": {
"better-sqlite3": "catalog:"
}
Comment on lines +82 to 84
Copy link

@coderabbitai coderabbitai bot Feb 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

better-sqlite3 is in dependencies instead of devDependencies, violating the bundled-CLI convention.

The coding guidelines and project convention require all runtime dependencies under devDependencies for apps that ship as bundled CLIs. Since better-sqlite3 is a native addon marked external in tsdown.config.ts, it genuinely can't be bundled — so placing it in dependencies is the correct choice for ensuring it's installed at the consumer side. However, this creates an implicit contract: every user who installs the published package must have a C++ toolchain (or a prebuilt binary must be available for their platform) because better-sqlite3 requires native compilation.

Consider whether this native dependency is acceptable for all target platforms, or if an optional/peer dependency with a graceful runtime fallback (which the code already has via bun:sqlite) would be less disruptive.

As per coding guidelines: "All projects under apps/ ship as bundled CLIs/binaries - list runtime dependencies in devDependencies (never dependencies) so the bundler owns the runtime payload."

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@apps/opencode/package.json` around lines 82 - 84, The package.json currently
lists better-sqlite3 under "dependencies" which violates the bundled-CLI
convention for apps in apps/; move "better-sqlite3" into "devDependencies" (or,
if you need consumers to optionally provide a native build, make it an
"optionalDependencies" or "peerDependencies") and keep tsdown.config.ts marking
it external; also ensure the runtime loader falls back to the existing
bun:sqlite path (the code path that conditions on better-sqlite3 vs bun:sqlite)
so the application gracefully handles platforms without a C++ toolchain.

}
240 changes: 183 additions & 57 deletions apps/opencode/src/data-loader.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,67 +1,43 @@
/**
* @fileoverview Data loading utilities for OpenCode usage analysis
*
* This module provides functions for loading and parsing OpenCode usage data
* from JSON message files stored in OpenCode data directories.
* OpenCode stores usage data in ~/.local/share/opencode/storage/message/
* This module provides functions for loading and parsing OpenCode usage data.
* OpenCode >= 1.2.2 stores data in a SQLite database at ~/.local/share/opencode/opencode.db
* Older versions stored data as JSON files in ~/.local/share/opencode/storage/message/
*
* @module data-loader
*/

import { readFile } from 'node:fs/promises';
import { existsSync } from 'node:fs';
import path from 'node:path';
import process from 'node:process';
import { isDirectorySync } from 'path-type';
import { glob } from 'tinyglobby';
import * as v from 'valibot';

/**
* Default OpenCode data directory path (~/.local/share/opencode)
*/
const DEFAULT_OPENCODE_PATH = '.local/share/opencode';

/**
* OpenCode storage subdirectory containing message data
*/
const OPENCODE_STORAGE_DIR_NAME = 'storage';

/**
* OpenCode messages subdirectory within storage
*/
const OPENCODE_MESSAGES_DIR_NAME = 'message';
const OPENCODE_SESSIONS_DIR_NAME = 'session';

/**
* Environment variable for specifying custom OpenCode data directory
*/
const OPENCODE_DB_FILENAME = 'opencode.db';
const OPENCODE_CONFIG_DIR_ENV = 'OPENCODE_DATA_DIR';

/**
* User home directory
*/
const USER_HOME_DIR = process.env.HOME ?? process.env.USERPROFILE ?? process.cwd();

/**
* Branded Valibot schema for model names
*/
const modelNameSchema = v.pipe(
v.string(),
v.minLength(1, 'Model name cannot be empty'),
v.brand('ModelName'),
);

/**
* Branded Valibot schema for session IDs
*/
const sessionIdSchema = v.pipe(
v.string(),
v.minLength(1, 'Session ID cannot be empty'),
v.brand('SessionId'),
);

/**
* OpenCode message token structure
*/
export const openCodeTokensSchema = v.object({
input: v.optional(v.number()),
output: v.optional(v.number()),
Expand All @@ -74,9 +50,6 @@ export const openCodeTokensSchema = v.object({
),
});

/**
* OpenCode message data structure
*/
export const openCodeMessageSchema = v.object({
id: v.string(),
sessionID: v.optional(sessionIdSchema),
Expand All @@ -98,9 +71,6 @@ export const openCodeSessionSchema = v.object({
directory: v.optional(v.string()),
});

/**
* Represents a single usage data entry loaded from OpenCode files
*/
export type LoadedUsageEntry = {
timestamp: Date;
sessionID: string;
Expand All @@ -122,10 +92,6 @@ export type LoadedSessionMetadata = {
directory: string;
};

/**
* Get OpenCode data directory
* @returns Path to OpenCode data directory, or null if not found
*/
export function getOpenCodePath(): string | null {
// Check environment variable first
const envPath = process.env[OPENCODE_CONFIG_DIR_ENV];
Expand All @@ -145,11 +111,166 @@ export function getOpenCodePath(): string | null {
return null;
}

/**
* Load OpenCode message from JSON file
* @param filePath - Path to message JSON file
* @returns Parsed message data or null on failure
*/
function getDbPath(openCodePath: string): string | null {
const dbPath = path.join(openCodePath, OPENCODE_DB_FILENAME);
return existsSync(dbPath) ? dbPath : null;
}

// ─── SQLite-based loading (OpenCode >= 1.2.2) ───────────────────────────────

const sqliteMessageDataSchema = v.object({
role: v.optional(v.string()),
providerID: v.optional(v.string()),
modelID: v.optional(v.string()),
tokens: v.optional(openCodeTokensSchema),
cost: v.optional(v.number()),
time: v.optional(
v.object({
created: v.optional(v.number()),
completed: v.optional(v.number()),
}),
),
});

interface SqliteRow {
[key: string]: unknown;
}

interface SqliteAdapter {
queryAll(sql: string): SqliteRow[];
close(): void;
}

function openSqliteDb(dbPath: string): SqliteAdapter {
try {
// eslint-disable-next-line @typescript-eslint/no-require-imports
const Database = require('better-sqlite3') as typeof import('better-sqlite3');
const db = new Database(dbPath, { readonly: true });
return {
queryAll(sql: string) {
return db.prepare(sql).all() as SqliteRow[];
},
close() {
db.close();
},
};
} catch {
// better-sqlite3 not available or failed to load
}

// eslint-disable-next-line @typescript-eslint/no-require-imports
const { Database } = require('bun:sqlite') as { Database: new (path: string, opts?: { readonly?: boolean }) => { query(sql: string): { all(): SqliteRow[] }; close(): void } };
const db = new Database(dbPath, { readonly: true });
return {
queryAll(sql: string) {
return db.query(sql).all();
},
close() {
db.close();
},
};
}
Comment on lines +144 to +172
Copy link

@coderabbitai coderabbitai bot Feb 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Overly broad catch in openSqliteDb — database errors are swallowed and misrouted to the bun:sqlite fallback.

Lines 157-159 catch all errors from better-sqlite3, including legitimate database errors (corrupt file, permission denied, locked DB). The intent is to fall back to bun:sqlite only when better-sqlite3 is not installed. If the module loads fine but the DB open fails, the error is silently swallowed and bun:sqlite is tried on the same corrupt/locked file — producing a confusing secondary error.

Narrow the catch to module-load failures only:

Proposed fix: separate module loading from DB opening 
 function openSqliteDb(dbPath: string): SqliteAdapter {
 	try {
 		// eslint-disable-next-line `@typescript-eslint/no-require-imports`
 		const Database = require('better-sqlite3') as typeof import('better-sqlite3');
+		const db = new Database(dbPath, { readonly: true });
+		return {
+			queryAll(sql: string) {
+				return db.prepare(sql).all() as SqliteRow[];
+			},
+			close() {
+				db.close();
+			},
+		};
+	} catch (e: unknown) {
+		if (!(e instanceof Error) || !('code' in e && e.code === 'MODULE_NOT_FOUND')) {
+			throw e;
+		}
-		const db = new Database(dbPath, { readonly: true });
-		return {
-			queryAll(sql: string) {
-				return db.prepare(sql).all() as SqliteRow[];
-			},
-			close() {
-				db.close();
-			},
-		};
-	} catch {
-		// better-sqlite3 not available or failed to load
 	}
 
 	// eslint-disable-next-line `@typescript-eslint/no-require-imports`
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@apps/opencode/src/data-loader.ts` around lines 144 - 172, The current
openSqliteDb function swallows all errors from the better-sqlite3 block, causing
real DB open errors to fall through to the bun:sqlite fallback; fix by splitting
module load from DB open: first try to require('better-sqlite3') inside a
try/catch and only if the require fails proceed to the bun fallback, but if
require succeeds then attempt new Database(dbPath, { readonly: true }) outside
that catch and let any errors opening the DB propagate (or rethrow) so they are
not mistaken for a missing module; update the code paths around the Database
variable and the returned queryAll/close implementations accordingly (refer to
function openSqliteDb and the require('better-sqlite3') / new Database(...)
usages).


function loadMessagesFromSqlite(dbPath: string): LoadedUsageEntry[] {
const db = openSqliteDb(dbPath);

try {
const rows = db.queryAll('SELECT id, session_id, time_created, data FROM message') as Array<{
id: string;
session_id: string;
time_created: number;
data: string;
}>;

const entries: LoadedUsageEntry[] = [];
const dedupeSet = new Set<string>();

for (const row of rows) {
if (dedupeSet.has(row.id)) {
continue;
}
dedupeSet.add(row.id);

let parsed: unknown;
try {
parsed = JSON.parse(row.data);
} catch {
continue;
}

const result = v.safeParse(sqliteMessageDataSchema, parsed);
if (!result.success) {
continue;
}

const data = result.output;

if (data.role !== 'assistant') {
continue;
}

if (data.tokens == null || (data.tokens.input === 0 && data.tokens.output === 0)) {
continue;
}

if (data.providerID == null || data.modelID == null) {
continue;
}

const createdMs = data.time?.created ?? row.time_created;

entries.push({
timestamp: new Date(createdMs),
sessionID: row.session_id,
usage: {
inputTokens: data.tokens.input ?? 0,
outputTokens: data.tokens.output ?? 0,
cacheCreationInputTokens: data.tokens.cache?.write ?? 0,
cacheReadInputTokens: data.tokens.cache?.read ?? 0,
},
model: data.modelID,
costUSD: data.cost ?? null,
});
}

return entries;
} finally {
db.close();
}
}

function loadSessionsFromSqlite(dbPath: string): Map<string, LoadedSessionMetadata> {
const db = openSqliteDb(dbPath);

try {
const rows = db.queryAll('SELECT id, project_id, parent_id, title, directory FROM session') as Array<{
id: string;
project_id: string;
parent_id: string | null;
title: string;
directory: string;
}>;

const sessionMap = new Map<string, LoadedSessionMetadata>();

for (const row of rows) {
sessionMap.set(row.id, {
id: row.id,
parentID: row.parent_id ?? null,
title: row.title || row.id,
projectID: row.project_id || 'unknown',
directory: row.directory || 'unknown',
});
}

return sessionMap;
} finally {
db.close();
}
}

// ─── Legacy JSON file-based loading (OpenCode < 1.2.2) ──────────────────────

async function loadOpenCodeMessage(
filePath: string,
): Promise<v.InferOutput<typeof openCodeMessageSchema> | null> {
Expand All @@ -162,11 +283,6 @@ async function loadOpenCodeMessage(
}
}

/**
* Convert OpenCode message to LoadedUsageEntry
* @param message - Parsed OpenCode message
* @returns LoadedUsageEntry suitable for aggregation
*/
function convertOpenCodeMessageToUsageEntry(
message: v.InferOutput<typeof openCodeMessageSchema>,
): LoadedUsageEntry {
Expand Down Expand Up @@ -216,6 +332,15 @@ export async function loadOpenCodeSessions(): Promise<Map<string, LoadedSessionM
return new Map();
}

const dbPath = getDbPath(openCodePath);
if (dbPath != null) {
try {
return loadSessionsFromSqlite(dbPath);
} catch {
// Fall through to legacy JSON loading
}
}
Comment on lines +335 to +342
Copy link

@coderabbitai coderabbitai bot Feb 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Silent catch blocks hide SQLite failures — users may unknowingly fall back to stale JSON data.

Both loadOpenCodeSessions (Line 339) and loadOpenCodeMessages (Line 385) catch all exceptions from the SQLite path and silently fall back to legacy JSON loading. If the DB exists but is corrupt, locked, or has a schema mismatch, the user gets stale/empty data with no indication of the problem.

Consider logging a warning before falling back so users can diagnose issues:

Proposed improvement: log on fallback 
 	if (dbPath != null) {
 		try {
 			return loadMessagesFromSqlite(dbPath);
-		} catch {
+		} catch (error) {
+			logger.warn('SQLite loading failed, falling back to JSON files', error);
 			// Fall through to legacy JSON loading
 		}
 	}

Note: The coding guidelines state "Do not use console.log - use logger.ts instead." Ensure you import from the project's logger utility.

Also applies to: 381-388

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@apps/opencode/src/data-loader.ts` around lines 335 - 342, Both
loadOpenCodeSessions and loadOpenCodeMessages silently swallow any exception
from the SQLite path (the getDbPath -> loadSessionsFromSqlite /
loadMessagesFromSqlite calls) and fall back to legacy JSON; change each catch to
log a warning (including the caught error.message/stack) via the project's
logger (import from logger.ts) before falling back so users can diagnose
corrupt/locked/schema issues. Locate the try/catch around loadSessionsFromSqlite
and loadMessagesFromSqlite, add a logger.warn or logger.error call that includes
context ("Falling back to JSON for OpenCode sessions/messages") plus the error,
and keep the existing fallback behavior. Ensure you add the logger import if
missing.


const sessionsDir = path.join(
openCodePath,
OPENCODE_STORAGE_DIR_NAME,
Expand Down Expand Up @@ -247,16 +372,21 @@ export async function loadOpenCodeSessions(): Promise<Map<string, LoadedSessionM
return sessionMap;
}

/**
* Load all OpenCode messages
* @returns Array of LoadedUsageEntry for aggregation
*/
export async function loadOpenCodeMessages(): Promise<LoadedUsageEntry[]> {
const openCodePath = getOpenCodePath();
if (openCodePath == null) {
return [];
}

const dbPath = getDbPath(openCodePath);
if (dbPath != null) {
try {
return loadMessagesFromSqlite(dbPath);
} catch {
// Fall through to legacy JSON loading
}
}

const messagesDir = path.join(
openCodePath,
OPENCODE_STORAGE_DIR_NAME,
Expand All @@ -267,7 +397,6 @@ export async function loadOpenCodeMessages(): Promise<LoadedUsageEntry[]> {
return [];
}

// Find all message JSON files
const messageFiles = await glob('**/*.json', {
cwd: messagesDir,
absolute: true,
Expand All @@ -283,17 +412,14 @@ export async function loadOpenCodeMessages(): Promise<LoadedUsageEntry[]> {
continue;
}

// Skip messages with no tokens
if (message.tokens == null || (message.tokens.input === 0 && message.tokens.output === 0)) {
continue;
}

// Skip if no provider or model
if (message.providerID == null || message.modelID == null) {
continue;
}

// Deduplicate by message ID
const dedupeKey = `${message.id}`;
if (dedupeSet.has(dedupeKey)) {
continue;
Expand Down
1 change: 1 addition & 0 deletions apps/opencode/tsdown.config.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,5 @@ export default defineConfig({
platform: 'node',
target: 'node20',
fixedExtension: false,
external: ['better-sqlite3'],
});
Loading