WeKnora deployment source with the reviewed wiki identity, summary and recovery fixes
Anydoc / anydoc parser engine (push) Canceled after 0s
App / Format, vet, test, and build (push) Canceled after 0s
CLI / test (macos-latest / go 1.26) (push) Canceled after 0s
CLI / test (ubuntu-latest / go 1.26) (push) Canceled after 0s
CLI / test (windows-latest / go 1.26) (push) Canceled after 0s
Build and Push Docker Image / build-ui (push) Canceled after 0s
Build and Push Docker Image / build-docreader (push) Canceled after 0s
Build and Push Docker Image / build-sandbox (push) Canceled after 0s
Build and Push Docker Image / build-app (amd64, linux/amd64, ubuntu-latest) (push) Canceled after 0s
Build and Push Docker Image / build-app (arm64, linux/arm64, ubuntu-24.04-arm) (push) Canceled after 0s
DocReader / Python tests and Go client tests (push) Canceled after 0s
Build & Publish (dsh-weknora) / Typecheck, test, pack (push) Canceled after 0s
Build & Publish (dsh-weknora) / End-to-end inside dsh (pinned) (push) Canceled after 0s
Build & Publish (dsh-weknora) / End-to-end inside dsh (latest, advisory) (push) Canceled after 0s
Frontend / Test, type-check, and build (push) Canceled after 0s
Go Lint Cache / warm lint cache (.) (push) Canceled after 0s
Go Lint Cache / warm lint cache (cli) (push) Canceled after 0s
Go Lint Cache / warm lint cache (client) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.10) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.11) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.12) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.13) (push) Canceled after 0s
Build and Push Docker Image / merge (push) Canceled after 0s
Build & Publish (dsh-weknora) / Publish to npm (push) Canceled after 0s
Build & Publish (mcp-server) / Build sdist + wheel (push) Canceled after 0s
Build & Publish (mcp-server) / Publish to PyPI (push) Canceled after 0s
Anydoc / anydoc parser engine (push) Canceled after 0s
App / Format, vet, test, and build (push) Canceled after 0s
CLI / test (macos-latest / go 1.26) (push) Canceled after 0s
CLI / test (ubuntu-latest / go 1.26) (push) Canceled after 0s
CLI / test (windows-latest / go 1.26) (push) Canceled after 0s
Build and Push Docker Image / build-ui (push) Canceled after 0s
Build and Push Docker Image / build-docreader (push) Canceled after 0s
Build and Push Docker Image / build-sandbox (push) Canceled after 0s
Build and Push Docker Image / build-app (amd64, linux/amd64, ubuntu-latest) (push) Canceled after 0s
Build and Push Docker Image / build-app (arm64, linux/arm64, ubuntu-24.04-arm) (push) Canceled after 0s
DocReader / Python tests and Go client tests (push) Canceled after 0s
Build & Publish (dsh-weknora) / Typecheck, test, pack (push) Canceled after 0s
Build & Publish (dsh-weknora) / End-to-end inside dsh (pinned) (push) Canceled after 0s
Build & Publish (dsh-weknora) / End-to-end inside dsh (latest, advisory) (push) Canceled after 0s
Frontend / Test, type-check, and build (push) Canceled after 0s
Go Lint Cache / warm lint cache (.) (push) Canceled after 0s
Go Lint Cache / warm lint cache (cli) (push) Canceled after 0s
Go Lint Cache / warm lint cache (client) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.10) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.11) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.12) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.13) (push) Canceled after 0s
Build and Push Docker Image / merge (push) Canceled after 0s
Build & Publish (dsh-weknora) / Publish to npm (push) Canceled after 0s
Build & Publish (mcp-server) / Build sdist + wheel (push) Canceled after 0s
Build & Publish (mcp-server) / Publish to PyPI (push) Canceled after 0s
This commit is contained in:
@@ -0,0 +1,267 @@
|
||||
# 添加新的网络搜索引擎
|
||||
|
||||
本文档说明如何在 WeKnora 中添加一个新的网络搜索引擎类型(如 Brave Search、Searx 等)。
|
||||
|
||||
## 架构概述
|
||||
|
||||
```
|
||||
internal/
|
||||
├── types/
|
||||
│ └── web_search_provider.go # 实体定义 + Provider 类型元数据
|
||||
├── infrastructure/
|
||||
│ └── web_search/
|
||||
│ ├── registry.go # Provider 工厂注册表
|
||||
│ ├── bing.go # Bing 实现
|
||||
│ ├── google.go # Google 实现
|
||||
│ ├── duckduckgo.go # DuckDuckGo 实现
|
||||
│ └── tavily.go # Tavily 实现
|
||||
├── container/
|
||||
│ └── container.go # DI 注册(registerWebSearchProviders)
|
||||
└── types/interfaces/
|
||||
└── web_search.go # WebSearchProvider 接口
|
||||
```
|
||||
|
||||
搜索引擎的 API 端点**硬编码**在代码中,不向用户暴露 BaseURL,从源头消除 SSRF 风险。
|
||||
|
||||
## 步骤
|
||||
|
||||
以添加 **Brave Search** 为例。
|
||||
|
||||
### 1. 在 `types/web_search_provider.go` 中注册类型常量
|
||||
|
||||
```go
|
||||
const (
|
||||
WebSearchProviderTypeBing WebSearchProviderType = "bing"
|
||||
WebSearchProviderTypeGoogle WebSearchProviderType = "google"
|
||||
WebSearchProviderTypeDuckDuckGo WebSearchProviderType = "duckduckgo"
|
||||
WebSearchProviderTypeTavily WebSearchProviderType = "tavily"
|
||||
WebSearchProviderTypeBrave WebSearchProviderType = "brave" // ← 新增
|
||||
)
|
||||
```
|
||||
|
||||
### 2. 在 `GetWebSearchProviderTypes()` 中添加类型元数据
|
||||
|
||||
```go
|
||||
{
|
||||
ID: "brave",
|
||||
Name: "Brave Search",
|
||||
Free: false,
|
||||
RequiresAPIKey: true,
|
||||
Description: "Brave Search API",
|
||||
DocsURL: "https://brave.com/search/api/",
|
||||
},
|
||||
```
|
||||
|
||||
字段说明:
|
||||
|
||||
| 字段 | 说明 |
|
||||
| ---------------- | ---------------------------------------------- |
|
||||
| `ID` | 唯一标识,存入数据库,不可更改 |
|
||||
| `Name` | 前端展示名称 |
|
||||
| `Free` | 是否免费 |
|
||||
| `RequiresAPIKey` | 是否需要 API Key |
|
||||
| `RequiresEngineID` | 是否需要额外 ID(如 Google CSE) |
|
||||
| `Description` | 简短描述 |
|
||||
| `DocsURL` | 官方文档链接,前端在添加对话框中显示 |
|
||||
|
||||
### 3. 创建 Provider 实现
|
||||
|
||||
新建 `internal/infrastructure/web_search/brave.go`:
|
||||
|
||||
```go
|
||||
package web_search
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/Tencent/WeKnora/internal/types/interfaces"
|
||||
)
|
||||
|
||||
const defaultBraveSearchURL = "https://api.search.brave.com/res/v1/web/search"
|
||||
|
||||
type BraveProvider struct {
|
||||
client *http.Client
|
||||
apiKey string
|
||||
}
|
||||
|
||||
// NewBraveProvider 从参数创建实例(不读环境变量)
|
||||
func NewBraveProvider(params types.WebSearchProviderParameters) (interfaces.WebSearchProvider, error) {
|
||||
if params.APIKey == "" {
|
||||
return nil, fmt.Errorf("API key is required for Brave provider")
|
||||
}
|
||||
return &BraveProvider{
|
||||
client: &http.Client{Timeout: 10 * time.Second},
|
||||
apiKey: params.APIKey,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func BraveProviderTypeInfo() types.WebSearchProviderTypeInfo {
|
||||
return types.WebSearchProviderTypeInfo{
|
||||
ID: "brave",
|
||||
Name: "Brave Search",
|
||||
Free: false,
|
||||
RequiresAPIKey: true,
|
||||
Description: "Brave Search API",
|
||||
DocsURL: "https://brave.com/search/api/",
|
||||
}
|
||||
}
|
||||
|
||||
func (p *BraveProvider) Name() string { return "brave" }
|
||||
|
||||
func (p *BraveProvider) Search(
|
||||
ctx context.Context, query string, maxResults int, includeDate bool,
|
||||
) ([]*types.WebSearchResult, error) {
|
||||
// 构造请求 — BaseURL 硬编码
|
||||
req, err := http.NewRequestWithContext(ctx, "GET",
|
||||
fmt.Sprintf("%s?q=%s&count=%d", defaultBraveSearchURL, query, maxResults), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("X-Subscription-Token", p.apiKey)
|
||||
req.Header.Set("Accept", "application/json")
|
||||
|
||||
resp, err := p.client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// 解析响应
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
var data braveResponse
|
||||
if err := json.Unmarshal(body, &data); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
results := make([]*types.WebSearchResult, 0, len(data.Web.Results))
|
||||
for _, r := range data.Web.Results {
|
||||
results = append(results, &types.WebSearchResult{
|
||||
Title: r.Title,
|
||||
URL: r.URL,
|
||||
Snippet: r.Description,
|
||||
Source: "brave",
|
||||
})
|
||||
}
|
||||
return results, nil
|
||||
}
|
||||
|
||||
type braveResponse struct {
|
||||
Web struct {
|
||||
Results []struct {
|
||||
Title string `json:"title"`
|
||||
URL string `json:"url"`
|
||||
Description string `json:"description"`
|
||||
} `json:"results"`
|
||||
} `json:"web"`
|
||||
}
|
||||
```
|
||||
|
||||
**关键要求**:
|
||||
|
||||
1. **构造函数签名**必须为 `func(types.WebSearchProviderParameters) (interfaces.WebSearchProvider, error)`
|
||||
2. **API 端点硬编码**为常量,不从参数中读取
|
||||
3. **实现 `interfaces.WebSearchProvider` 接口**:`Name()` 和 `Search()`
|
||||
|
||||
### 4. 在 Service 的参数校验中添加新类型
|
||||
|
||||
编辑 `internal/application/service/web_search_provider.go`:
|
||||
|
||||
```go
|
||||
func isValidProviderType(provider types.WebSearchProviderType) bool {
|
||||
switch provider {
|
||||
case types.WebSearchProviderTypeBing,
|
||||
types.WebSearchProviderTypeGoogle,
|
||||
types.WebSearchProviderTypeDuckDuckGo,
|
||||
types.WebSearchProviderTypeTavily,
|
||||
types.WebSearchProviderTypeBrave: // ← 新增
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 5. 在 DI 容器中注册
|
||||
|
||||
编辑 `internal/container/container.go` 的 `registerWebSearchProviders` 函数:
|
||||
|
||||
```go
|
||||
func registerWebSearchProviders(registry *infra_web_search.Registry) {
|
||||
// ... 已有注册 ...
|
||||
|
||||
// Register Brave provider type
|
||||
registry.Register(infra_web_search.BraveProviderTypeInfo(), infra_web_search.NewBraveProvider)
|
||||
}
|
||||
```
|
||||
|
||||
### 6. 验证
|
||||
|
||||
```bash
|
||||
# 编译
|
||||
go build ./...
|
||||
|
||||
# 启动后调用 API 验证类型列表
|
||||
curl http://localhost:8080/api/v1/web-search-providers/types \
|
||||
-H 'X-API-Key: your_key'
|
||||
|
||||
# 创建 Brave 搜索引擎实例
|
||||
curl -X POST http://localhost:8080/api/v1/web-search-providers \
|
||||
-H 'X-API-Key: your_key' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"name": "Brave Search",
|
||||
"provider": "brave",
|
||||
"parameters": { "api_key": "BSA..." },
|
||||
"is_default": true
|
||||
}'
|
||||
```
|
||||
|
||||
## 需要额外参数的情况
|
||||
|
||||
如果新引擎需要 API Key 以外的参数(类似 Google 的 `engine_id`),有两种方式:
|
||||
|
||||
### 方式一:使用 `ExtraConfig`
|
||||
|
||||
利用 `WebSearchProviderParameters.ExtraConfig` 字段,不需要改类型定义:
|
||||
|
||||
```go
|
||||
func NewFooProvider(params types.WebSearchProviderParameters) (interfaces.WebSearchProvider, error) {
|
||||
region := params.ExtraConfig["region"]
|
||||
if region == "" {
|
||||
region = "us"
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
前端在 `GetWebSearchProviderTypes()` 中可以标注需要哪些 extra 字段(后续支持动态表单渲染)。
|
||||
|
||||
### 方式二:添加专用字段
|
||||
|
||||
如果参数非常通用(比如多个引擎都需要),可以在 `WebSearchProviderParameters` 中添加新字段:
|
||||
|
||||
```go
|
||||
type WebSearchProviderParameters struct {
|
||||
APIKey string `json:"api_key,omitempty"`
|
||||
EngineID string `json:"engine_id,omitempty"`
|
||||
Region string `json:"region,omitempty"` // ← 新增
|
||||
ExtraConfig map[string]string `json:"extra_config,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
同时在 `WebSearchProviderTypeInfo` 中添加 `RequiresRegion bool` 等字段,前端根据此动态显示输入框。
|
||||
|
||||
## 文件变更清单
|
||||
|
||||
| 文件 | 操作 |
|
||||
| ---- | ---- |
|
||||
| `internal/types/web_search_provider.go` | 添加常量 + 类型元数据 |
|
||||
| `internal/infrastructure/web_search/brave.go` | **新建** Provider 实现 |
|
||||
| `internal/application/service/web_search_provider.go` | `isValidProviderType` 加新类型 |
|
||||
| `internal/container/container.go` | `registerWebSearchProviders` 注册 |
|
||||
Reference in New Issue
Block a user