feat: add Anthropic format conversion for chat runs and improvements (#347)
* fix: improve chat compression and tool display Context Compression Fixes: - Remove duplicate token calculation in compress() - Simplify compress() to only execute compression, not judge - Add buildConversationHistory() to preserve tool calls in LLM context - Remove unused estimateMessagesTokens() and contextLength parameter - Move all judgment logic to chat-run-socket.ts (uses accurate DB tokens) Tool Call Display Improvements: - Add tool execution duration display (format: 1.272s) - Add success/error status icons with circular backgrounds - Replace text error with SVG icon (X in red circle) - Replace old checkmark with polished green checkmark icon - Add i18n key 'chat.executionDuration' for all locales Bug Fixes: - Fix streaming-indicator stuck by adding try-finally in handleEvent - Add debug logging for compression flow diagnosis - Fix template syntax error in MessageList.vue Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(chat): convert conversation history to Anthropic format before sending to Gateway - Add convertToAnthropicFormat() to transform OpenAI format to Anthropic format - Handle DeepSeek reasoning_content in thinking blocks - Properly convert tool_use and tool_result blocks - Add convertFromAnthropicFormat() for parsing SSE responses - Handle stringified Python arrays in resume messages - Record debug history files for troubleshooting (original vs converted) - Fix tool_call_id validation to prevent empty ID errors - Clean internal Hermes fields (call_id, response_item_id) from tool_calls Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(chat): optimize message parsing and add debug logging - Only check for stringified arrays in assistant messages (performance) - Improve parsing error handling: keep original content on parse failure - Add debug logging for upstream events (reasoning/thinking tracking) - Log run.completed event keys for troubleshooting Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(chat): add message pagination and reasoning sync improvements **Message Pagination:** - Add getSessionDetailPaginated() for paginated message loading - Query with DESC order then reverse in code for optimal performance - Remove listSessionsPaginated() (not needed) **Reasoning Sync:** - Add bidirectional reasoning merge in syncFromHermes - Memory → DB: preserve streamed reasoning from SSE events - DB → Memory: restore reasoning if Hermes Gateway fixes storage - Send resumed event after sync completes with complete messages - Fix reasoning field inconsistency: use unified 'reasoning' field **Message Parsing:** - Only parse stringified arrays for assistant messages (performance) - Improve parse error handling: keep original content on failure - Add debug logging for upstream reasoning/thinking events **Bug Fixes:** - Fix reasoning content display: now works on both SSE and resume - Ensure reasoning is preserved across page refreshes via sync + resumed event Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: increase default pagination limit for messages to 500 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: remove auto-resumed event trigger and clean up debug code - Remove automatic resumed event trigger in syncFromHermes to avoid timing issues - Clean up unused imports (fs, join) - Remove debug history file logging code - Fix socket parameter passing in handleAbort, markCompleted, and syncFromHermes - Change usage emit from room broadcast to socket-only emit - Remove console.log debug statement Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: use reasoning field in convertToAnthropicFormat Change convertToAnthropicFormat to read from reasoning field instead of reasoning_content for consistency with database schema and frontend. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: parse stringified array content and improve logs - Parse stringified array format in run.completed to extract thinking/text/tool_use - Send parsed content to frontend via parsed_content/parsed_reasoning/parsed_tool_calls - Frontend updates last assistant message with parsed content - Remove ellipsis from log messages, show full content - Add detailed logging for conversion and parsing Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: move finalOutputTrimmed outside else block * fix(chat): handle double-serialized content in resumeSession - Remove outer quotes before parsing stringified array format - Updated changelog for v0.5.2 and v0.5.3 with multilingual support - Fixed message pagination with DESC query + array reverse Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(chat): improve error logging for resume parsing - Add detailed logging for double-serialized content parsing - Log content preview when parsing fails to diagnose issues Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * revert(chat): use simple Python-to-JSON replacement - Revert to simple .replace(/'/g, '"') approach - Parsing failures will keep original content as-is Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
ekko
@@ -2,16 +2,21 @@
|
||||
* Sync Hermes sessions from all profiles on startup.
|
||||
* Reads api_server sessions from Hermes state.db and imports into local DB.
|
||||
* Only runs when local DB is empty (first startup).
|
||||
*
|
||||
* Uses sessions-db.ts query logic to properly aggregate session chains.
|
||||
*/
|
||||
import { readdirSync, existsSync } from 'fs'
|
||||
import { resolve, join } from 'path'
|
||||
import { homedir } from 'os'
|
||||
import { DatabaseSync } from 'node:sqlite'
|
||||
import { randomBytes } from 'crypto'
|
||||
import { getProfileDir } from './hermes-profile'
|
||||
import { createSession, addMessage, updateSession, getSession } from '../../db/hermes/session-store'
|
||||
import { createSession, addMessage, updateSession } from '../../db/hermes/session-store'
|
||||
import { getDb } from '../../db/index'
|
||||
import { logger } from '../logger'
|
||||
import { listSessionSummaries as listHermesSessionSummaries } from '../../db/hermes/sessions-db'
|
||||
|
||||
const HERMES_BASE = resolve(homedir(), '.hermes')
|
||||
const PROFILES_DIR = join(HERMES_BASE, 'profiles')
|
||||
|
||||
/**
|
||||
* Generate a UUID v4 without external dependencies
|
||||
@@ -29,45 +34,6 @@ function generateUuid(): string {
|
||||
].join('-')
|
||||
}
|
||||
|
||||
const HERMES_BASE = resolve(homedir(), '.hermes')
|
||||
const PROFILES_DIR = join(HERMES_BASE, 'profiles')
|
||||
|
||||
interface HermesSessionRow {
|
||||
id: string
|
||||
source: string
|
||||
model: string
|
||||
title: string | null
|
||||
started_at: number
|
||||
ended_at: number | null
|
||||
end_reason: string | null
|
||||
message_count: number
|
||||
tool_call_count: number
|
||||
input_tokens: number
|
||||
output_tokens: number
|
||||
cache_read_tokens: number
|
||||
cache_write_tokens: number
|
||||
reasoning_tokens: number
|
||||
estimated_cost_usd: number
|
||||
last_active: number
|
||||
}
|
||||
|
||||
interface HermesMessageRow {
|
||||
id: number | string
|
||||
session_id: string
|
||||
role: string
|
||||
content: string
|
||||
tool_call_id: string | null
|
||||
tool_calls: any[] | null
|
||||
tool_name: string | null
|
||||
timestamp: number
|
||||
token_count: number | null
|
||||
finish_reason: string | null
|
||||
reasoning: string | null
|
||||
reasoning_details: string | null
|
||||
reasoning_content: string | null
|
||||
codex_reasoning_items: string | null
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all available profile names including 'default'
|
||||
*/
|
||||
@@ -85,163 +51,85 @@ function getAllProfiles(): string[] {
|
||||
}
|
||||
|
||||
/**
|
||||
* Open Hermes state.db for a specific profile
|
||||
* Sync api_server sessions from a single profile.
|
||||
* Uses sessions-db.ts query logic to properly aggregate session chains.
|
||||
*/
|
||||
function openHermesStateDb(profile: string): DatabaseSync {
|
||||
const profileDir = getProfileDir(profile)
|
||||
const dbPath = join(profileDir, 'state.db')
|
||||
|
||||
if (!existsSync(dbPath)) {
|
||||
throw new Error(`Hermes state.db not found for profile '${profile}' at ${dbPath}`)
|
||||
}
|
||||
|
||||
return new DatabaseSync(dbPath, { readOnly: true })
|
||||
}
|
||||
|
||||
/**
|
||||
* Sync api_server sessions from a single profile
|
||||
*/
|
||||
function syncProfileSessions(profile: string): {
|
||||
async function syncProfileSessions(profile: string): Promise<{
|
||||
synced: number
|
||||
skipped: number
|
||||
errors: string[]
|
||||
} {
|
||||
const result = { synced: 0, skipped: 0, errors: [] as string[] }
|
||||
}> {
|
||||
const result = { synced: 0, errors: [] as string[] }
|
||||
|
||||
try {
|
||||
const db = openHermesStateDb(profile)
|
||||
// Use listSessionSummaries to get aggregated session chains
|
||||
// This returns only root sessions with aggregated stats from the entire chain
|
||||
const summaries = await listHermesSessionSummaries('api_server', 10000, profile)
|
||||
|
||||
try {
|
||||
// Check if sessions table has estimated_cost_usd column
|
||||
const tableInfo = db.prepare('PRAGMA table_info(sessions)').all() as Array<{ name: string }>
|
||||
const hasEstimatedCost = tableInfo.some(col => col.name === 'estimated_cost_usd')
|
||||
logger.info(`[session-sync] profile '${profile}': found ${summaries.length} aggregated session chains`)
|
||||
|
||||
// Build SELECT query - only include estimated_cost_usd if column exists
|
||||
const estimatedCostCol = hasEstimatedCost ? ', COALESCE(estimated_cost_usd, 0) AS estimated_cost_usd' : ', 0 AS estimated_cost_usd'
|
||||
for (const hermesSession of summaries) {
|
||||
try {
|
||||
// Generate new session ID for local DB
|
||||
const newSessionId = generateUuid()
|
||||
|
||||
// Get all api_server sessions
|
||||
const sessions = db.prepare(`
|
||||
SELECT
|
||||
id,
|
||||
source,
|
||||
COALESCE(model, '') AS model,
|
||||
title,
|
||||
started_at,
|
||||
ended_at,
|
||||
end_reason,
|
||||
message_count,
|
||||
tool_call_count,
|
||||
input_tokens,
|
||||
output_tokens,
|
||||
cache_read_tokens,
|
||||
cache_write_tokens,
|
||||
reasoning_tokens${estimatedCostCol}
|
||||
FROM sessions
|
||||
WHERE source = 'api_server'
|
||||
ORDER BY started_at ASC
|
||||
`).all() as unknown as Omit<HermesSessionRow, 'preview' | 'last_active'>[]
|
||||
// Create session in local DB
|
||||
createSession({
|
||||
id: newSessionId,
|
||||
profile,
|
||||
model: hermesSession.model,
|
||||
title: hermesSession.title || undefined,
|
||||
})
|
||||
|
||||
logger.info(`[session-sync] profile '${profile}': found ${sessions.length} api_server sessions`)
|
||||
for (const hermesSession of sessions) {
|
||||
try {
|
||||
// Check if this Hermes session ID already exists in local DB
|
||||
const existing = getSession(hermesSession.id)
|
||||
if (existing) {
|
||||
result.skipped++
|
||||
continue
|
||||
}
|
||||
// Get full detail including all messages from the session chain
|
||||
const { getSessionDetailFromDbWithProfile } = await import('../../db/hermes/sessions-db')
|
||||
const detail = await getSessionDetailFromDbWithProfile(hermesSession.id, profile)
|
||||
|
||||
// Generate new session ID
|
||||
const newSessionId = generateUuid()
|
||||
|
||||
// Create session in local DB
|
||||
createSession({
|
||||
id: newSessionId,
|
||||
profile,
|
||||
model: hermesSession.model,
|
||||
title: hermesSession.title || undefined,
|
||||
})
|
||||
|
||||
// Get all messages for this session
|
||||
const messages = db.prepare(`
|
||||
SELECT
|
||||
id,
|
||||
session_id,
|
||||
role,
|
||||
content,
|
||||
tool_call_id,
|
||||
tool_calls,
|
||||
tool_name,
|
||||
timestamp,
|
||||
token_count,
|
||||
finish_reason,
|
||||
reasoning,
|
||||
reasoning_details,
|
||||
reasoning_content,
|
||||
codex_reasoning_items
|
||||
FROM messages
|
||||
WHERE session_id = ?
|
||||
ORDER BY timestamp, id
|
||||
`).all(hermesSession.id) as unknown as HermesMessageRow[]
|
||||
|
||||
// Insert all messages
|
||||
for (const msg of messages) {
|
||||
addMessage({
|
||||
session_id: newSessionId,
|
||||
role: msg.role,
|
||||
content: msg.content,
|
||||
tool_call_id: msg.tool_call_id,
|
||||
tool_calls: msg.tool_calls,
|
||||
tool_name: msg.tool_name,
|
||||
timestamp: msg.timestamp,
|
||||
token_count: msg.token_count,
|
||||
finish_reason: msg.finish_reason,
|
||||
reasoning: msg.reasoning,
|
||||
reasoning_details: msg.reasoning_details,
|
||||
reasoning_content: msg.reasoning_content,
|
||||
codex_reasoning_items: msg.codex_reasoning_items,
|
||||
})
|
||||
}
|
||||
|
||||
// Generate preview from first user message
|
||||
const firstUserMessage = messages.find(m => m.role === 'user' && m.content)
|
||||
let preview = ''
|
||||
if (firstUserMessage && firstUserMessage.content) {
|
||||
// Remove newlines, truncate to 63 chars
|
||||
preview = firstUserMessage.content
|
||||
.replace(/[\n\r]/g, ' ')
|
||||
.trim()
|
||||
.slice(0, 63)
|
||||
}
|
||||
|
||||
// Update session with Hermes data
|
||||
const estimatedCost = typeof hermesSession.estimated_cost_usd === 'number'
|
||||
? hermesSession.estimated_cost_usd
|
||||
: 0
|
||||
|
||||
updateSession(newSessionId, {
|
||||
started_at: hermesSession.started_at,
|
||||
ended_at: hermesSession.ended_at,
|
||||
end_reason: hermesSession.end_reason,
|
||||
input_tokens: hermesSession.input_tokens,
|
||||
output_tokens: hermesSession.output_tokens,
|
||||
cache_read_tokens: hermesSession.cache_read_tokens,
|
||||
cache_write_tokens: hermesSession.cache_write_tokens,
|
||||
reasoning_tokens: hermesSession.reasoning_tokens,
|
||||
estimated_cost_usd: estimatedCost,
|
||||
last_active: hermesSession.started_at, // Use started_at as fallback since last_active doesn't exist in Hermes state.db
|
||||
preview,
|
||||
})
|
||||
|
||||
result.synced++
|
||||
logger.info(`[session-sync] synced Hermes session ${hermesSession.id} -> ${newSessionId} (${messages.length} messages)`)
|
||||
} catch (err: any) {
|
||||
result.errors.push(`session ${hermesSession.id}: ${err.message}`)
|
||||
logger.warn(err, `[session-sync] failed to sync session ${hermesSession.id}`)
|
||||
if (!detail || !detail.messages) {
|
||||
result.errors.push(`session ${hermesSession.id}: failed to load messages`)
|
||||
logger.warn(`[session-sync] failed to load messages for session ${hermesSession.id}`)
|
||||
continue
|
||||
}
|
||||
|
||||
// Insert all messages from the entire chain
|
||||
for (const msg of detail.messages) {
|
||||
addMessage({
|
||||
session_id: newSessionId,
|
||||
role: msg.role,
|
||||
content: msg.content,
|
||||
tool_call_id: msg.tool_call_id,
|
||||
tool_calls: msg.tool_calls,
|
||||
tool_name: msg.tool_name,
|
||||
timestamp: msg.timestamp,
|
||||
token_count: msg.token_count,
|
||||
finish_reason: msg.finish_reason,
|
||||
reasoning: msg.reasoning,
|
||||
reasoning_details: msg.reasoning_details,
|
||||
reasoning_content: msg.reasoning_content,
|
||||
codex_reasoning_items: msg.codex_reasoning_items,
|
||||
})
|
||||
}
|
||||
|
||||
// Update session with aggregated stats from Hermes
|
||||
updateSession(newSessionId, {
|
||||
started_at: hermesSession.started_at,
|
||||
ended_at: hermesSession.ended_at,
|
||||
end_reason: hermesSession.end_reason,
|
||||
input_tokens: hermesSession.input_tokens,
|
||||
output_tokens: hermesSession.output_tokens,
|
||||
cache_read_tokens: hermesSession.cache_read_tokens,
|
||||
cache_write_tokens: hermesSession.cache_write_tokens,
|
||||
reasoning_tokens: hermesSession.reasoning_tokens,
|
||||
estimated_cost_usd: hermesSession.estimated_cost_usd,
|
||||
last_active: hermesSession.last_active,
|
||||
preview: hermesSession.preview,
|
||||
})
|
||||
|
||||
result.synced++
|
||||
logger.info(`[session-sync] synced Hermes session ${hermesSession.id} -> ${newSessionId} (${detail.messages.length} messages, thread_session_count=${detail.thread_session_count})`)
|
||||
} catch (err: any) {
|
||||
result.errors.push(`session ${hermesSession.id}: ${err.message}`)
|
||||
logger.warn(err, `[session-sync] failed to sync session ${hermesSession.id}`)
|
||||
}
|
||||
} finally {
|
||||
db.close()
|
||||
}
|
||||
} catch (err: any) {
|
||||
if (!err.message.includes('state.db not found')) {
|
||||
@@ -257,7 +145,7 @@ function syncProfileSessions(profile: string): {
|
||||
* Main entry point: sync all profiles on startup
|
||||
* Only runs if local DB is empty (first startup or after DB reset)
|
||||
*/
|
||||
export function syncAllHermesSessionsOnStartup(): void {
|
||||
export async function syncAllHermesSessionsOnStartup(): Promise<void> {
|
||||
// Check if local DB has any sessions - only sync if completely empty
|
||||
const db = getDb()
|
||||
if (!db) {
|
||||
@@ -279,13 +167,11 @@ export function syncAllHermesSessionsOnStartup(): void {
|
||||
logger.info(`[session-sync] found ${profiles.length} profiles: ${profiles.join(', ')}`)
|
||||
|
||||
let totalSynced = 0
|
||||
let totalSkipped = 0
|
||||
let totalErrors = 0
|
||||
|
||||
for (const profile of profiles) {
|
||||
const result = syncProfileSessions(profile)
|
||||
const result = await syncProfileSessions(profile)
|
||||
totalSynced += result.synced
|
||||
totalSkipped += result.skipped
|
||||
totalErrors += result.errors.length
|
||||
|
||||
if (result.errors.length > 0) {
|
||||
@@ -299,5 +185,5 @@ export function syncAllHermesSessionsOnStartup(): void {
|
||||
}
|
||||
}
|
||||
|
||||
logger.info(`[session-sync] sync complete: synced=${totalSynced}, skipped=${totalSkipped}, errors=${totalErrors}`)
|
||||
logger.info(`[session-sync] sync complete: synced=${totalSynced}, errors=${totalErrors}`)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user