feat(copilot): integrate GitHub Copilot provider with dynamic model list / 集成 GitHub Copilot provider 与动态模型列表 (#239)

* feat(copilot): integrate GitHub Copilot provider with dynamic model list 集成 GitHub Copilot provider 与动态模型列表 EN: - New copilot-models service: fetch live model list from GitHub /models API - Filter noise IDs (accounts/, text-embedding, rerank prefixes) - Pass through preview/disabled metadata to frontend - Cache isolated per OAuth token (FNV-1a hash key) to prevent cross-account leak - Multi-source token resolution: env > apps.json > gh CLI - ModelSelector renders PREVIEW (orange) and UNAVAILABLE (gray, non-selectable) badges with tooltips - ProviderFormModal exposes Copilot OAuth login entry - New CopilotLoginModal component: guides gh auth login device flow - ProviderCard hides delete button for OAuth-only builtin providers (copilot/codex/nous) since their credentials live outside auth.json ZH: - 新增 copilot-models 服务：从 GitHub /models live API 拉取模型列表 - 噪音 ID 过滤（accounts/、text-embedding、rerank 前缀） - preview/disabled 元数据透传至前端 - 缓存按 OAuth token 隔离（FNV-1a hash key），避免切换 profile 串账号 - 多源 token 解析优先级：env > apps.json > gh CLI - ModelSelector 渲染 PREVIEW（橙色）/ UNAVAILABLE（灰色、不可选）badge，附 tooltip - ProviderFormModal 提供 Copilot OAuth 登录入口 - 新增 CopilotLoginModal 组件：引导 gh auth login 设备流程 - ProviderCard 对 OAuth-only builtin（copilot/codex/nous）隐藏删除按钮其凭证不在 auth.json，删除按钮原本无效 Tests / 测试: new copilot-models suite (cache isolation, noise filter, preview/disabled passthrough) + copilot-login-modal — 24/24 passed. Pre-existing sessions-db-lineage failure on upstream/main is unrelated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(copilot): switch to explicit opt-in per maintainer feedback 回应 PR #239 review：上一版会自动把系统级 GitHub OAuth 凭证（VS Code Copilot 插件、gh CLI 登录态）当作 hermes provider 拉到列表里，对未在 hermes 中注册过 Copilot 的用户造成困扰。本次改为显式 opt-in：用户必须通过 Add Provider 主动添加，删除时按 token 来源决定是否清 ~/.hermes/.env，并避免误清理 VS Code / gh CLI 用户的全局凭证。 Address PR #239 review feedback. Previously Copilot would silently appear in the provider list whenever the host had any GitHub OAuth token (VS Code plugin, gh CLI login). This caused confusion for users who never explicitly registered Copilot in hermes. Now Copilot requires explicit opt-in via Add Provider; on delete we only clear ~/.hermes/.env when the token actually originated there, leaving VS Code / gh CLI credentials untouched. What changed - 新增 ~/.hermes-web-ui/config.json 的 copilotEnabled flag 控制可见性 - 即便能解析到 token，未启用时也不在列表中显示 - resolveCopilotOAuthTokenWithSource 区分 token 来源（env / gh-cli / apps-json） - ProviderFormModal 增加 GitHub Copilot 入口；无 token 时进 device flow modal - CopilotLoginModal 重写为 in-app device flow 状态机（不再要求用户在终端跑 gh） - 删除 Copilot 时仅 source='env' 才清 ~/.hermes/.env，并自动 fallback 默认模型 - 老用户升级兼容：若 default 仍指向已禁用的 copilot，后端清空 default 让前端兜底 API - POST /api/hermes/copilot-auth/check-token - POST /api/hermes/copilot-auth/enable - POST /api/hermes/copilot-auth/disable - POST /api/hermes/copilot-auth/start (device flow) - POST /api/hermes/copilot-auth/poll (device flow) Tests - tests/server/copilot-auth-controller.test.ts (11 cases) - tests/server/copilot-device-flow.test.ts (12 cases) - tests/client/copilot-login-modal.test.ts 重写覆盖状态机 Follow-ups (留作后续 PR) - device flow session 未绑定 profile，登录中切 profile 会写到错的 .env - copilot device-code 接口的 expires_in 字段未使用，硬编码 15 分钟超时 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-26 22:51:35 +08:00
parent b07a8fc76f
commit 610f3eb9d0
30 changed files with 2264 additions and 16 deletions
@@ -0,0 +1,237 @@
+import { randomUUID } from 'crypto'
+import { startDeviceFlow, pollDeviceFlow } from '../../services/hermes/copilot-device-flow'
+import { saveEnvValue, readConfigYaml, writeConfigYaml } from '../../services/config-helpers'
+import {
+  invalidateAllCaches,
+  resolveCopilotOAuthTokenWithSource,
+  type CopilotTokenSource,
+} from '../../services/hermes/copilot-models'
+import { getActiveEnvPath } from '../../services/hermes/hermes-profile'
+import { readAppConfig, writeAppConfig } from '../../services/app-config'
+import { readFile } from 'fs/promises'
+import { logger } from '../../services/logger'
+
+const POLL_MAX_DURATION_MS = 15 * 60 * 1000 // 15 minutes hard ceiling
+const SESSION_GC_GRACE_MS = 60 * 1000
+
+interface CopilotLoginSession {
+  id: string
+  deviceCode: string
+  userCode: string
+  verificationUrl: string
+  expiresIn: number
+  interval: number
+  status: 'pending' | 'approved' | 'denied' | 'expired' | 'error'
+  error?: string
+  createdAt: number
+}
+
+const sessions = new Map<string, CopilotLoginSession>()
+
+function cleanupSessions(): void {
+  const now = Date.now()
+  sessions.forEach((s, id) => {
+    if (now - s.createdAt > POLL_MAX_DURATION_MS + SESSION_GC_GRACE_MS) {
+      sessions.delete(id)
+    }
+  })
+}
+
+async function persistToken(token: string): Promise<void> {
+  // 与 disable 对称：只动 ~/.hermes/.env，apps.json 是 VS Code 的文件不要碰。
+  // 同时把 enabled 置 true —— device flow 完成后用户已显式同意启用 Copilot。
+  // NOTE: 故意不写 process.env.COPILOT_GITHUB_TOKEN —— 否则该值会跨 profile 持续覆盖
+  // resolveCopilotOAuthTokenWithSource 的 .env 读取，导致切到别的 profile 仍解析到当前
+  // profile 的 token。invalidateAllCaches() + .env 文件本身已能保证下次解析读到新 token。
+  await saveEnvValue('COPILOT_GITHUB_TOKEN', token)
+  await writeAppConfig({ copilotEnabled: true })
+  invalidateAllCaches()
+}
+
+async function readEnvContent(): Promise<string> {
+  try { return await readFile(getActiveEnvPath(), 'utf-8') } catch { return '' }
+}
+
+async function loginWorker(session: CopilotLoginSession): Promise<void> {
+  const startTime = Date.now()
+  let interval = Math.max(1, session.interval) * 1000
+
+  while (Date.now() - startTime < POLL_MAX_DURATION_MS) {
+    await new Promise((resolve) => setTimeout(resolve, interval))
+    if (session.status !== 'pending') return
+
+    const result = await pollDeviceFlow(session.deviceCode)
+
+    if (result.kind === 'success') {
+      try {
+        await persistToken(result.access_token)
+        session.status = 'approved'
+        logger.info('Copilot OAuth login successful')
+      } catch (err: any) {
+        logger.error(err, 'Copilot OAuth: failed to persist token')
+        session.status = 'error'
+        session.error = err?.message ?? 'failed to persist token'
+      }
+      return
+    }
+
+    if (result.kind === 'pending') continue
+    if (result.kind === 'slow_down') {
+      interval += 5000
+      continue
+    }
+    if (result.kind === 'denied') {
+      session.status = 'denied'
+      return
+    }
+    if (result.kind === 'expired') {
+      session.status = 'expired'
+      return
+    }
+    logger.error('Copilot OAuth poll error: %s %s', result.error, result.description ?? '')
+    session.status = 'error'
+    session.error = result.description ?? result.error
+    return
+  }
+
+  session.status = 'expired'
+}
+
+export async function start(ctx: any): Promise<void> {
+  cleanupSessions()
+  try {
+    const data = await startDeviceFlow()
+    const sessionId = randomUUID()
+    const session: CopilotLoginSession = {
+      id: sessionId,
+      deviceCode: data.device_code,
+      userCode: data.user_code,
+      verificationUrl: data.verification_uri,
+      expiresIn: data.expires_in,
+      interval: data.interval,
+      status: 'pending',
+      createdAt: Date.now(),
+    }
+    sessions.set(sessionId, session)
+
+    loginWorker(session).catch((err) => {
+      logger.error(err, 'Copilot login worker error')
+      session.status = 'error'
+      session.error = err?.message ?? String(err)
+    })
+
+    ctx.body = {
+      session_id: sessionId,
+      user_code: data.user_code,
+      verification_url: data.verification_uri,
+      expires_in: data.expires_in,
+      interval: data.interval,
+    }
+  } catch (err: any) {
+    logger.error(err, 'Copilot OAuth start failed')
+    if (err?.name === 'TimeoutError' || err?.name === 'AbortError') {
+      ctx.status = 504
+      ctx.body = { error: 'GitHub timeout' }
+      return
+    }
+    ctx.status = 502
+    ctx.body = { error: err?.message ?? 'GitHub OAuth start failed' }
+  }
+}
+
+export async function poll(ctx: any): Promise<void> {
+  const session = sessions.get(ctx.params.sessionId)
+  if (!session) {
+    ctx.status = 404
+    ctx.body = { error: 'Session not found' }
+    return
+  }
+  ctx.body = { status: session.status, error: session.error || null }
+}
+
+/**
+ * Reports current token resolution and whether Copilot is opt-in enabled.
+ * Frontend Add Provider flow uses this to decide whether to show the
+ * "token detected, click Add" confirmation or kick off device flow.
+ *
+ * Side effect: invalidates the model list cache so a subsequent listing
+ * picks up gh-cli logout / VS Code sign-out without server restart.
+ */
+export async function checkToken(ctx: any): Promise<void> {
+  invalidateAllCaches()
+  const env = await readEnvContent()
+  const { token, source } = await resolveCopilotOAuthTokenWithSource(env)
+  const cfg = await readAppConfig()
+  ctx.body = {
+    has_token: Boolean(token),
+    source: source as CopilotTokenSource,
+    enabled: cfg.copilotEnabled === true,
+  }
+}
+
+export async function enable(ctx: any): Promise<void> {
+  await writeAppConfig({ copilotEnabled: true })
+  invalidateAllCaches()
+  ctx.body = { ok: true }
+}
+
+/**
+ * "Soft delete" Copilot from the web-ui provider list.
+ *   - Always: copilotEnabled = false (hides provider regardless of token source).
+ *   - source='env'        → also clear ~/.hermes/.env COPILOT_GITHUB_TOKEN
+ *                           (this token belongs to the hermes ecosystem).
+ *   - source='gh-cli'     → leave gh CLI alone (user's terminal sessions).
+ *   - source='apps-json'  → leave VS Code Copilot plugin alone.
+ * The user can re-add Copilot any time via "Add Provider".
+ */
+export async function disable(ctx: any): Promise<void> {
+  const env = await readEnvContent()
+  const { source } = await resolveCopilotOAuthTokenWithSource(env)
+
+  // 步骤 1：先清掉默认模型（最容易失败的一步：写 yaml 可能失败）。
+  // 不能 swallow —— 否则会出现 "list 已隐藏 copilot 但 default 仍是 copilot" 的中间态。
+  let clearedDefault = false
+  try {
+    const cfg = await readConfigYaml()
+    const modelSection = cfg.model
+    if (typeof modelSection === 'object' && modelSection !== null) {
+      const provider = String(modelSection.provider || '').trim().toLowerCase()
+      if (provider === 'copilot') {
+        cfg.model = {}
+        await writeConfigYaml(cfg)
+        clearedDefault = true
+      }
+    }
+  } catch (err: any) {
+    logger.error(err, 'Copilot disable failed: cannot clear default model')
+    ctx.status = 500
+    ctx.body = { error: `failed to clear default model: ${err?.message ?? 'unknown error'}` }
+    return
+  }
+
+  // 步骤 2：清 .env（仅当 source='env'）。失败也不能让 enabled flag 偷偷置 false。
+  try {
+    if (source === 'env') {
+      await saveEnvValue('COPILOT_GITHUB_TOKEN', '')
+      delete process.env.COPILOT_GITHUB_TOKEN
+    }
+  } catch (err: any) {
+    logger.error(err, 'Copilot disable failed: cannot clear .env')
+    ctx.status = 500
+    ctx.body = { error: `failed to clear .env: ${err?.message ?? 'unknown error'}` }
+    return
+  }
+
+  // 步骤 3：最后翻 enabled flag。前两步成功才执行。
+  try {
+    await writeAppConfig({ copilotEnabled: false })
+    invalidateAllCaches()
+  } catch (err: any) {
+    logger.error(err, 'Copilot disable failed: cannot persist enabled flag')
+    ctx.status = 500
+    ctx.body = { error: `failed to persist enabled flag: ${err?.message ?? 'unknown error'}` }
+    return
+  }
+
+  ctx.body = { ok: true, cleared_env: source === 'env', cleared_default: clearedDefault }
+}
@@ -3,9 +3,17 @@ import { existsSync, readFileSync } from 'fs'
 import { getActiveEnvPath, getActiveAuthPath } from '../../services/hermes/hermes-profile'
 import { readConfigYaml, writeConfigYaml, fetchProviderModels, buildModelGroups, PROVIDER_ENV_MAP } from '../../services/config-helpers'
 import { buildProviderModelMap, PROVIDER_PRESETS } from '../../shared/providers'
+import { getCopilotModelsDetailed, resolveCopilotOAuthToken, type CopilotModelMeta } from '../../services/hermes/copilot-models'
+import { readAppConfig } from '../../services/app-config'

 const PROVIDER_MODEL_CATALOG = buildProviderModelMap()

+// Copilot 授权检测：复用同一套 token 解析逻辑（含 ~/.config/github-copilot/apps.json
+// 与 ghp_ PAT 跳过），与 getCopilotModels 行为一致，避免出现"模型能拉到却被判未授权"。
+async function isCopilotAuthorized(envContent: string): Promise<boolean> {
+  return !!(await resolveCopilotOAuthToken(envContent))
+}
+
 export async function getAvailable(ctx: any) {
  try {
    const config = await readConfigYaml()
@@ -31,7 +39,7 @@ export async function getAvailable(ctx: any) {
      currentDefault = modelSection.trim()
    }

-    const groups: Array<{ provider: string; label: string; base_url: string; models: string[]; api_key: string }> = []
+    const groups: Array<{ provider: string; label: string; base_url: string; models: string[]; api_key: string; model_meta?: Record<string, { preview?: boolean; disabled?: boolean }> }> = []
    const seenProviders = new Set<string>()

    let envContent = ''
@@ -47,10 +55,10 @@ export async function getAvailable(ctx: any) {
      const match = envContent.match(new RegExp(`^${key}\\s*=\\s*(.+)`, 'm'))
      return match?.[1]?.trim() || ''
    }
-    const addGroup = (provider: string, label: string, base_url: string, models: string[], api_key: string) => {
+    const addGroup = (provider: string, label: string, base_url: string, models: string[], api_key: string, model_meta?: Record<string, { preview?: boolean; disabled?: boolean }>) => {
      if (seenProviders.has(provider)) return
      seenProviders.add(provider)
-      groups.push({ provider, label, base_url, models: [...models], api_key })
+      groups.push({ provider, label, base_url, models: [...models], api_key, ...(model_meta ? { model_meta } : {}) })
    }

    const isOAuthAuthorized = (providerKey: string): boolean => {
@@ -69,9 +77,39 @@ export async function getAvailable(ctx: any) {
      } catch { return false }
    }

+    // 同一请求内复用 copilot 动态模型（getCopilotModelsDetailed 内部有 inflight + 缓存，
+    // 这里再缓存到局部变量进一步减少分支）
+    let copilotLiveModels: CopilotModelMeta[] | null = null
+    const getCopilotLive = async (): Promise<CopilotModelMeta[]> => {
+      if (copilotLiveModels !== null) return copilotLiveModels
+      try { copilotLiveModels = await getCopilotModelsDetailed(envContent) }
+      catch { copilotLiveModels = [] }
+      return copilotLiveModels
+    }
+
+    // Copilot 显式 opt-in：即便能解析到 token，未通过 web-ui Add Provider 显式启用
+    // 时也不返回。避免误把 VS Code/gh CLI 用户的全局凭证当作 hermes provider。
+    const appConfig = await readAppConfig()
+    const copilotEnabled = appConfig.copilotEnabled === true
+
+    // 兼容老用户：上一版本会"自动 fallback discovery"出 Copilot；升级后这些用户的
+    // config.yaml 可能仍把 model.default 指向某个 copilot 模型。若此时 copilot 已不
+    // 启用，把返回的 default 清掉，让前端兜底自动选剩余 provider 的第一个 model。
+    if (!copilotEnabled && currentDefaultProvider.toLowerCase() === 'copilot') {
+      currentDefault = ''
+      currentDefaultProvider = ''
+    }
+
    for (const [providerKey, envMapping] of Object.entries(PROVIDER_ENV_MAP)) {
      if (envMapping.api_key_env && !envHasValue(envMapping.api_key_env)) continue
-      if (!envMapping.api_key_env && !isOAuthAuthorized(providerKey)) continue
+      if (!envMapping.api_key_env) {
+        if (providerKey === 'copilot') {
+          if (!copilotEnabled) continue
+          if (!(await isCopilotAuthorized(envContent))) continue
+        } else if (!isOAuthAuthorized(providerKey)) {
+          continue
+        }
+      }
      const preset = PROVIDER_PRESETS.find((p: any) => p.value === providerKey)
      const label = preset?.label || providerKey.replace(/^custom:/, '')
      let baseUrl = preset?.base_url || ''
@@ -79,9 +117,27 @@ export async function getAvailable(ctx: any) {
        baseUrl = envGetValue(envMapping.base_url_env) || baseUrl
      }
      const catalogModels = PROVIDER_MODEL_CATALOG[providerKey]
-      if (catalogModels && catalogModels.length > 0) {
+      let modelsList: string[] = catalogModels && catalogModels.length > 0 ? [...catalogModels] : []
+      let modelMeta: Record<string, { preview?: boolean; disabled?: boolean }> | undefined
+      if (providerKey === 'copilot') {
+        const live = await getCopilotLive()
+        if (live.length > 0) {
+          modelsList = live.map((m) => m.id)
+          modelMeta = {}
+          for (const m of live) {
+            if (m.preview || m.disabled) {
+              modelMeta[m.id] = {
+                ...(m.preview ? { preview: true } : {}),
+                ...(m.disabled ? { disabled: true } : {}),
+              }
+            }
+          }
+          if (Object.keys(modelMeta).length === 0) modelMeta = undefined
+        }
+      }
+      if (modelsList.length > 0) {
        const apiKey = envMapping.api_key_env ? envGetValue(envMapping.api_key_env) : ''
-        addGroup(providerKey, label, baseUrl, catalogModels, apiKey)
+        addGroup(providerKey, label, baseUrl, modelsList, apiKey, modelMeta)
      }
    }

@@ -115,15 +171,25 @@ export async function getAvailable(ctx: any) {

    for (const g of groups) { g.models = Array.from(new Set(g.models)) }

+    // 动态拉一次 copilot 模型用于 allProviders 展示（同一请求复用缓存）
+    // 未启用 Copilot 时跳过拉取，避免空跑网络请求。
+    const liveCopilotModels = copilotEnabled ? await getCopilotLive() : []
+    const liveCopilotIds = liveCopilotModels.map((m) => m.id)
+
+    const allProvidersBase = PROVIDER_PRESETS.map((p: any) => ({
+      provider: p.value,
+      label: p.label,
+      base_url: p.base_url,
+      models: p.value === 'copilot' && liveCopilotIds.length > 0 ? liveCopilotIds : p.models,
+    }))
+
    if (groups.length === 0) {
      const fallback = buildModelGroups(config)
-      const allProviders = PROVIDER_PRESETS.map((p: any) => ({ provider: p.value, label: p.label, base_url: p.base_url, models: p.models }))
-      ctx.body = { ...fallback, allProviders }
+      ctx.body = { ...fallback, allProviders: allProvidersBase }
      return
    }

-    const allProviders = PROVIDER_PRESETS.map((p: any) => ({ provider: p.value, label: p.label, base_url: p.base_url, models: p.models }))
-    ctx.body = { default: currentDefault, default_provider: currentDefaultProvider, groups, allProviders }
+    ctx.body = { default: currentDefault, default_provider: currentDefaultProvider, groups, allProviders: allProvidersBase }
  } catch (err: any) {
    ctx.status = 500
    ctx.body = { error: err.message }