WeKnora deployment source with the reviewed wiki identity, summary and recovery fixes
Anydoc / anydoc parser engine (push) Canceled after 0s
App / Format, vet, test, and build (push) Canceled after 0s
CLI / test (macos-latest / go 1.26) (push) Canceled after 0s
CLI / test (ubuntu-latest / go 1.26) (push) Canceled after 0s
CLI / test (windows-latest / go 1.26) (push) Canceled after 0s
Build and Push Docker Image / build-ui (push) Canceled after 0s
Build and Push Docker Image / build-docreader (push) Canceled after 0s
Build and Push Docker Image / build-sandbox (push) Canceled after 0s
Build and Push Docker Image / build-app (amd64, linux/amd64, ubuntu-latest) (push) Canceled after 0s
Build and Push Docker Image / build-app (arm64, linux/arm64, ubuntu-24.04-arm) (push) Canceled after 0s
DocReader / Python tests and Go client tests (push) Canceled after 0s
Build & Publish (dsh-weknora) / Typecheck, test, pack (push) Canceled after 0s
Build & Publish (dsh-weknora) / End-to-end inside dsh (pinned) (push) Canceled after 0s
Build & Publish (dsh-weknora) / End-to-end inside dsh (latest, advisory) (push) Canceled after 0s
Frontend / Test, type-check, and build (push) Canceled after 0s
Go Lint Cache / warm lint cache (.) (push) Canceled after 0s
Go Lint Cache / warm lint cache (cli) (push) Canceled after 0s
Go Lint Cache / warm lint cache (client) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.10) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.11) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.12) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.13) (push) Canceled after 0s
Build and Push Docker Image / merge (push) Canceled after 0s
Build & Publish (dsh-weknora) / Publish to npm (push) Canceled after 0s
Build & Publish (mcp-server) / Build sdist + wheel (push) Canceled after 0s
Build & Publish (mcp-server) / Publish to PyPI (push) Canceled after 0s
Anydoc / anydoc parser engine (push) Canceled after 0s
App / Format, vet, test, and build (push) Canceled after 0s
CLI / test (macos-latest / go 1.26) (push) Canceled after 0s
CLI / test (ubuntu-latest / go 1.26) (push) Canceled after 0s
CLI / test (windows-latest / go 1.26) (push) Canceled after 0s
Build and Push Docker Image / build-ui (push) Canceled after 0s
Build and Push Docker Image / build-docreader (push) Canceled after 0s
Build and Push Docker Image / build-sandbox (push) Canceled after 0s
Build and Push Docker Image / build-app (amd64, linux/amd64, ubuntu-latest) (push) Canceled after 0s
Build and Push Docker Image / build-app (arm64, linux/arm64, ubuntu-24.04-arm) (push) Canceled after 0s
DocReader / Python tests and Go client tests (push) Canceled after 0s
Build & Publish (dsh-weknora) / Typecheck, test, pack (push) Canceled after 0s
Build & Publish (dsh-weknora) / End-to-end inside dsh (pinned) (push) Canceled after 0s
Build & Publish (dsh-weknora) / End-to-end inside dsh (latest, advisory) (push) Canceled after 0s
Frontend / Test, type-check, and build (push) Canceled after 0s
Go Lint Cache / warm lint cache (.) (push) Canceled after 0s
Go Lint Cache / warm lint cache (cli) (push) Canceled after 0s
Go Lint Cache / warm lint cache (client) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.10) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.11) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.12) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.13) (push) Canceled after 0s
Build and Push Docker Image / merge (push) Canceled after 0s
Build & Publish (dsh-weknora) / Publish to npm (push) Canceled after 0s
Build & Publish (mcp-server) / Build sdist + wheel (push) Canceled after 0s
Build & Publish (mcp-server) / Publish to PyPI (push) Canceled after 0s
This commit is contained in:
@@ -0,0 +1,92 @@
|
||||
package skills
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
)
|
||||
|
||||
// SkillEnvResolver produces the environment one execution gets. It is separate
|
||||
// from SkillSource because the values are per-caller: the same skill, in the
|
||||
// same image, hands a different key to a different Principal. The implementation
|
||||
// lives in the service layer, which can reach the repository.
|
||||
//
|
||||
// It is keyed by skill NAME rather than id because every path into the manager
|
||||
// is name-addressed: ExecuteScript receives the name the model wrote, and the
|
||||
// row id is an implementation detail of the installed-skill source.
|
||||
type SkillEnvResolver interface {
|
||||
// ResolveEnv returns the values to inject and the names of any required
|
||||
// variable that neither the admin nor this caller has filled in. An empty
|
||||
// skillName asks for the caller's config-wide variables alone. The caller's
|
||||
// identity is taken from ctx, never from a parameter.
|
||||
ResolveEnv(ctx context.Context, skillName string) (env map[string]string, missing []string, err error)
|
||||
}
|
||||
|
||||
// MissingSkillEnvError reports that execution was refused because a required
|
||||
// variable has no value at either layer. It is typed so the agent loop can
|
||||
// relay a sentence a person can act on instead of the KeyError or 401 the
|
||||
// script would otherwise produce.
|
||||
type MissingSkillEnvError struct {
|
||||
SkillName string
|
||||
Names []string
|
||||
}
|
||||
|
||||
func (e *MissingSkillEnvError) Error() string {
|
||||
// English, like every other error in this codebase: the agent relays this
|
||||
// to the user and translates it into whatever language they are speaking.
|
||||
// execute_skill_script has no env parameter, so it cannot take a value the
|
||||
// user just typed. shell_exec can: naming the skill and passing the value
|
||||
// in env runs the command and stores the value for the next run. Pointing
|
||||
// at the settings page alone would strand IM users, who have no such page.
|
||||
return fmt.Sprintf(
|
||||
"skill %q needs the environment variable(s) %s, which nobody has set yet. "+
|
||||
"Ask the user for them, then run the skill through shell_exec with "+
|
||||
"skill_name=%q and the values in env — they are stored for that user "+
|
||||
"afterwards. They can also be set under Settings → Sandbox secrets.",
|
||||
e.SkillName, strings.Join(e.Names, ", "), e.SkillName,
|
||||
)
|
||||
}
|
||||
|
||||
// ApplyResolvedEnv overlays resolved onto env WITHOUT displacing anything env
|
||||
// already carries. It is exported because shell_exec applies the same overlay
|
||||
// for its optional skill_name parameter and must not diverge from it.
|
||||
//
|
||||
// This is the second layer of reserved-name protection. Task 2's write-time
|
||||
// blacklist is the first, but a value written before that blacklist existed
|
||||
// would still be in the database, and letting it land on
|
||||
// WEKNORA_SKILL_OUTPUT_DIR would silently redirect the turn's artifacts to a
|
||||
// directory nobody drains. Skipping existing keys makes that impossible
|
||||
// regardless of what is stored.
|
||||
func ApplyResolvedEnv(env, resolved map[string]string) {
|
||||
for name, value := range resolved {
|
||||
if _, taken := env[name]; taken {
|
||||
continue
|
||||
}
|
||||
env[name] = value
|
||||
}
|
||||
}
|
||||
|
||||
// applySessionPackagePath prepends the per-session extra-packages directory
|
||||
// so a frozen skill venv can still see `pip install --target` extras without
|
||||
// mutating the snapshot. Missing directories are ignored by the interpreters.
|
||||
func applySessionPackagePath(env map[string]string, skillName string) {
|
||||
if env == nil {
|
||||
return
|
||||
}
|
||||
dir := sandbox.SessionSkillPackageDir(skillName)
|
||||
if dir == "" {
|
||||
return
|
||||
}
|
||||
prependPathEnv(env, pythonPathEnvVar, dir)
|
||||
prependPathEnv(env, nodePathEnvVar, dir)
|
||||
}
|
||||
|
||||
func prependPathEnv(env map[string]string, key, dir string) {
|
||||
if existing := strings.TrimSpace(env[key]); existing != "" {
|
||||
env[key] = dir + ":" + existing
|
||||
return
|
||||
}
|
||||
env[key] = dir
|
||||
}
|
||||
@@ -0,0 +1,151 @@
|
||||
package skills
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// The runtime seeds the artifact directory variables before the resolver runs.
|
||||
// Whatever a user stored, those keys must survive untouched: they are how the
|
||||
// turn's products are collected.
|
||||
func TestApplyResolvedEnvAddsNewKeysAndKeepsExistingOnes(t *testing.T) {
|
||||
env := map[string]string{
|
||||
artifactOutputEnvVar: "/workspace/output",
|
||||
artifactHistoryEnvVar: "/workspace/output",
|
||||
}
|
||||
|
||||
ApplyResolvedEnv(env, map[string]string{
|
||||
"TAVILY_API_KEY": "user-key",
|
||||
artifactOutputEnvVar: "/tmp/hijacked",
|
||||
artifactHistoryEnvVar: "/tmp/hijacked",
|
||||
})
|
||||
|
||||
require.Equal(t, "user-key", env["TAVILY_API_KEY"])
|
||||
require.Equal(t, "/workspace/output", env[artifactOutputEnvVar])
|
||||
require.Equal(t, "/workspace/output", env[artifactHistoryEnvVar])
|
||||
}
|
||||
|
||||
func TestApplySessionPackagePathPrependsPythonAndNodePath(t *testing.T) {
|
||||
env := map[string]string{pythonPathEnvVar: "/already"}
|
||||
|
||||
applySessionPackagePath(env, "律师助手")
|
||||
|
||||
dir := sandbox.SessionSkillPackageDir("律师助手")
|
||||
require.Equal(t, dir+":/already", env[pythonPathEnvVar])
|
||||
require.Equal(t, dir, env[nodePathEnvVar])
|
||||
}
|
||||
|
||||
// The message is read by the agent and relayed verbatim to a person, so it has
|
||||
// to name the skill, the variables, and where to go and fix it.
|
||||
func TestMissingSkillEnvErrorMessageNamesSkillVarsAndWhereToSetThem(t *testing.T) {
|
||||
err := error(&MissingSkillEnvError{
|
||||
SkillName: "web-search",
|
||||
Names: []string{"TAVILY_API_KEY", "SERP_TOKEN"},
|
||||
})
|
||||
|
||||
msg := err.Error()
|
||||
require.Contains(t, msg, "web-search")
|
||||
require.Contains(t, msg, "TAVILY_API_KEY")
|
||||
require.Contains(t, msg, "SERP_TOKEN")
|
||||
require.Contains(t, msg, "Sandbox secrets")
|
||||
|
||||
var typed *MissingSkillEnvError
|
||||
require.True(t, errors.As(err, &typed))
|
||||
require.Equal(t, "web-search", typed.SkillName)
|
||||
require.Equal(t, []string{"TAVILY_API_KEY", "SERP_TOKEN"}, typed.Names)
|
||||
}
|
||||
|
||||
// stubEnvResolver records what ExecuteScript asked it for.
|
||||
type stubEnvResolver struct {
|
||||
env map[string]string
|
||||
missing []string
|
||||
err error
|
||||
|
||||
calls []string
|
||||
}
|
||||
|
||||
func (r *stubEnvResolver) ResolveEnv(
|
||||
_ context.Context, skillName string,
|
||||
) (map[string]string, []string, error) {
|
||||
r.calls = append(r.calls, skillName)
|
||||
return r.env, r.missing, r.err
|
||||
}
|
||||
|
||||
func TestExecuteScriptBlocksOnMissingRequiredEnv(t *testing.T) {
|
||||
dir := preloadedSkillDir(t, "web-search", "search the web")
|
||||
sandboxMgr := &recordingSandboxManager{}
|
||||
resolver := &stubEnvResolver{missing: []string{"TAVILY_API_KEY"}}
|
||||
mgr := NewManager(&ManagerConfig{SkillDirs: []string{dir}, Enabled: true}, sandboxMgr).
|
||||
WithEnvResolver(resolver)
|
||||
require.NoError(t, mgr.Initialize(context.Background()))
|
||||
|
||||
result, err := mgr.ExecuteScript(context.Background(), "web-search", "scripts/run.py", nil, "")
|
||||
|
||||
require.Nil(t, result)
|
||||
var missing *MissingSkillEnvError
|
||||
require.ErrorAs(t, err, &missing)
|
||||
require.Equal(t, "web-search", missing.SkillName)
|
||||
require.Equal(t, []string{"TAVILY_API_KEY"}, missing.Names)
|
||||
// The whole point of blocking is that the script never runs.
|
||||
require.Zero(t, sandboxMgr.calls)
|
||||
require.Nil(t, sandboxMgr.config)
|
||||
}
|
||||
|
||||
func TestExecuteScriptPropagatesResolverError(t *testing.T) {
|
||||
dir := preloadedSkillDir(t, "web-search", "search the web")
|
||||
sandboxMgr := &recordingSandboxManager{}
|
||||
boom := errors.New("load user envs: connection refused")
|
||||
mgr := NewManager(&ManagerConfig{SkillDirs: []string{dir}, Enabled: true}, sandboxMgr).
|
||||
WithEnvResolver(&stubEnvResolver{err: boom})
|
||||
require.NoError(t, mgr.Initialize(context.Background()))
|
||||
|
||||
_, err := mgr.ExecuteScript(context.Background(), "web-search", "scripts/run.py", nil, "")
|
||||
|
||||
require.ErrorIs(t, err, boom)
|
||||
require.Zero(t, sandboxMgr.calls)
|
||||
}
|
||||
|
||||
func TestExecuteScriptInjectsResolvedEnvOnceKeyedBySkillName(t *testing.T) {
|
||||
dir := preloadedSkillDir(t, "web-search", "search the web")
|
||||
sandboxMgr := &recordingSandboxManager{}
|
||||
resolver := &stubEnvResolver{env: map[string]string{
|
||||
"TAVILY_API_KEY": "user-key",
|
||||
artifactOutputEnvVar: "/tmp/hijacked",
|
||||
}}
|
||||
mgr := NewManager(&ManagerConfig{SkillDirs: []string{dir}, Enabled: true}, sandboxMgr).
|
||||
WithEnvResolver(resolver)
|
||||
require.NoError(t, mgr.Initialize(context.Background()))
|
||||
|
||||
_, err := mgr.ExecuteScript(context.Background(), "web-search", "scripts/run.py", nil, "")
|
||||
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, []string{"web-search"}, resolver.calls)
|
||||
require.Equal(t, 1, sandboxMgr.calls)
|
||||
require.NotNil(t, sandboxMgr.config)
|
||||
require.Equal(t, "user-key", sandboxMgr.config.Env["TAVILY_API_KEY"])
|
||||
require.Equal(t, "/workspace/output", sandboxMgr.config.Env[artifactOutputEnvVar])
|
||||
}
|
||||
|
||||
// Without a resolver the execution path must stay exactly as it was.
|
||||
func TestExecuteScriptWithoutResolverInjectsNothingExtra(t *testing.T) {
|
||||
dir := preloadedSkillDir(t, "web-search", "search the web")
|
||||
sandboxMgr := &recordingSandboxManager{}
|
||||
mgr := NewManager(&ManagerConfig{SkillDirs: []string{dir}, Enabled: true}, sandboxMgr)
|
||||
require.NoError(t, mgr.Initialize(context.Background()))
|
||||
|
||||
_, err := mgr.ExecuteScript(context.Background(), "web-search", "scripts/run.py", nil, "")
|
||||
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, 1, sandboxMgr.calls)
|
||||
injected := map[string]bool{}
|
||||
for _, name := range InjectedSandboxEnvVars() {
|
||||
injected[name] = true
|
||||
}
|
||||
for key := range sandboxMgr.config.Env {
|
||||
require.True(t, injected[key], "injected env %q is not in InjectedSandboxEnvVars", key)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,161 @@
|
||||
package skills
|
||||
|
||||
import (
|
||||
"context"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestExampleSkillsIntegration tests with the actual example skills in examples/skills
|
||||
func TestExampleSkillsIntegration(t *testing.T) {
|
||||
// Get the path to examples/skills relative to this test file
|
||||
_, filename, _, ok := runtime.Caller(0)
|
||||
if !ok {
|
||||
t.Fatal("Failed to get current file path")
|
||||
}
|
||||
|
||||
// Navigate from internal/agent/skills to examples/skills
|
||||
skillsDir := filepath.Join(filepath.Dir(filename), "..", "..", "..", "examples", "skills")
|
||||
|
||||
// Create loader
|
||||
loader := NewLoader([]string{skillsDir})
|
||||
|
||||
// Discover skills
|
||||
metadata, err := loader.DiscoverSkills()
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to discover skills: %v", err)
|
||||
}
|
||||
|
||||
if len(metadata) == 0 {
|
||||
t.Skip("No example skills found in examples/skills directory")
|
||||
}
|
||||
|
||||
t.Logf("Discovered %d example skills:", len(metadata))
|
||||
for _, m := range metadata {
|
||||
t.Logf(" - %s: %s", m.Name, truncate(m.Description, 60))
|
||||
}
|
||||
|
||||
// Test loading the pdf-processing skill
|
||||
pdfSkill, err := loader.LoadSkillInstructions("pdf-processing")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to load pdf-processing skill: %v", err)
|
||||
}
|
||||
|
||||
// Verify metadata
|
||||
if pdfSkill.Name != "pdf-processing" {
|
||||
t.Errorf("Expected name 'pdf-processing', got '%s'", pdfSkill.Name)
|
||||
}
|
||||
|
||||
if pdfSkill.Instructions == "" {
|
||||
t.Error("Expected instructions to be non-empty")
|
||||
}
|
||||
|
||||
t.Logf("PDF Processing skill instructions length: %d characters", len(pdfSkill.Instructions))
|
||||
|
||||
// Test loading additional file (FORMS.md)
|
||||
formsFile, err := loader.LoadSkillFile("pdf-processing", "FORMS.md")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to load FORMS.md: %v", err)
|
||||
}
|
||||
|
||||
if formsFile.Content == "" {
|
||||
t.Error("Expected FORMS.md content to be non-empty")
|
||||
}
|
||||
|
||||
t.Logf("FORMS.md content length: %d characters", len(formsFile.Content))
|
||||
|
||||
// Test loading script
|
||||
scriptFile, err := loader.LoadSkillFile("pdf-processing", "scripts/analyze_form.py")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to load analyze_form.py: %v", err)
|
||||
}
|
||||
|
||||
if !scriptFile.IsScript {
|
||||
t.Error("analyze_form.py should be marked as script")
|
||||
}
|
||||
|
||||
t.Logf("analyze_form.py content length: %d characters", len(scriptFile.Content))
|
||||
|
||||
// Test list files
|
||||
files, err := loader.ListSkillFiles("pdf-processing")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to list skill files: %v", err)
|
||||
}
|
||||
|
||||
t.Logf("Files in pdf-processing skill:")
|
||||
for _, f := range files {
|
||||
t.Logf(" - %s (script: %v)", f, IsScript(f))
|
||||
}
|
||||
}
|
||||
|
||||
// TestManagerWithExampleSkills tests the Manager with example skills
|
||||
func TestManagerWithExampleSkills(t *testing.T) {
|
||||
// Get the path to examples/skills
|
||||
_, filename, _, ok := runtime.Caller(0)
|
||||
if !ok {
|
||||
t.Fatal("Failed to get current file path")
|
||||
}
|
||||
|
||||
skillsDir := filepath.Join(filepath.Dir(filename), "..", "..", "..", "examples", "skills")
|
||||
|
||||
// Create manager
|
||||
config := &ManagerConfig{
|
||||
SkillDirs: []string{skillsDir},
|
||||
AllowedSkills: []string{}, // Allow all
|
||||
Enabled: true,
|
||||
}
|
||||
|
||||
manager := NewManager(config, nil)
|
||||
|
||||
// Initialize
|
||||
ctx := context.Background()
|
||||
if err := manager.Initialize(ctx); err != nil {
|
||||
t.Fatalf("Failed to initialize manager: %v", err)
|
||||
}
|
||||
|
||||
// Get metadata for system prompt
|
||||
metadata := manager.GetAllMetadata()
|
||||
if len(metadata) == 0 {
|
||||
t.Skip("No example skills found")
|
||||
}
|
||||
|
||||
t.Logf("Manager discovered %d skills for system prompt injection", len(metadata))
|
||||
|
||||
// Simulate what the agent would do:
|
||||
// 1. First, get metadata (Level 1 - already in system prompt)
|
||||
for _, m := range metadata {
|
||||
t.Logf("Level 1 (metadata): %s - %s", m.Name, truncate(m.Description, 50))
|
||||
}
|
||||
|
||||
// 2. When user request matches, load full skill instructions (Level 2)
|
||||
skill, err := manager.LoadSkill(ctx, "pdf-processing")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to load skill: %v", err)
|
||||
}
|
||||
|
||||
t.Logf("Level 2 (instructions): Loaded %d characters of instructions", len(skill.Instructions))
|
||||
|
||||
// 3. If skill references additional files, read them (Level 3)
|
||||
formsContent, err := manager.ReadSkillFile(ctx, "pdf-processing", "FORMS.md")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to read skill file: %v", err)
|
||||
}
|
||||
|
||||
t.Logf("Level 3 (resources): Loaded FORMS.md with %d characters", len(formsContent))
|
||||
|
||||
// Test GetSkillInfo
|
||||
info, err := manager.GetSkillInfo(ctx, "pdf-processing")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to get skill info: %v", err)
|
||||
}
|
||||
|
||||
t.Logf("Skill info: name=%s, files=%d", info.Name, len(info.Files))
|
||||
}
|
||||
|
||||
func truncate(s string, maxLen int) string {
|
||||
if len(s) <= maxLen {
|
||||
return s
|
||||
}
|
||||
return s[:maxLen] + "..."
|
||||
}
|
||||
@@ -0,0 +1,308 @@
|
||||
package skills
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Loader handles skill discovery and loading from the filesystem
|
||||
// It implements the Progressive Disclosure pattern by separating
|
||||
// metadata discovery (Level 1) from instructions loading (Level 2/3)
|
||||
type Loader struct {
|
||||
// skillDirs are the directories to search for skills
|
||||
skillDirs []string
|
||||
// discoveredSkills caches discovered skill metadata
|
||||
discoveredSkills map[string]*Skill
|
||||
}
|
||||
|
||||
// NewLoader creates a new skill loader with the specified search directories
|
||||
func NewLoader(skillDirs []string) *Loader {
|
||||
return &Loader{
|
||||
skillDirs: skillDirs,
|
||||
discoveredSkills: make(map[string]*Skill),
|
||||
}
|
||||
}
|
||||
|
||||
// DiscoverSkills scans all configured directories for SKILL.md files
|
||||
// and extracts their metadata (Level 1). This is a lightweight operation
|
||||
// that only reads the frontmatter of each skill file.
|
||||
func (l *Loader) DiscoverSkills() ([]*SkillMetadata, error) {
|
||||
var allMetadata []*SkillMetadata
|
||||
|
||||
for _, dir := range l.skillDirs {
|
||||
metadata, err := l.discoverInDirectory(dir)
|
||||
if err != nil {
|
||||
// Log warning but continue with other directories
|
||||
continue
|
||||
}
|
||||
allMetadata = append(allMetadata, metadata...)
|
||||
}
|
||||
|
||||
return allMetadata, nil
|
||||
}
|
||||
|
||||
// discoverInDirectory scans a single directory for skill subdirectories
|
||||
func (l *Loader) discoverInDirectory(dir string) ([]*SkillMetadata, error) {
|
||||
var metadata []*SkillMetadata
|
||||
|
||||
// Check if directory exists
|
||||
info, err := os.Stat(dir)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil, nil // Directory doesn't exist, skip silently
|
||||
}
|
||||
return nil, fmt.Errorf("failed to access skill directory %s: %w", dir, err)
|
||||
}
|
||||
|
||||
if !info.IsDir() {
|
||||
return nil, fmt.Errorf("%s is not a directory", dir)
|
||||
}
|
||||
|
||||
// Read directory entries
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read skill directory %s: %w", dir, err)
|
||||
}
|
||||
|
||||
for _, entry := range entries {
|
||||
if !entry.IsDir() {
|
||||
continue
|
||||
}
|
||||
|
||||
skillPath := filepath.Join(dir, entry.Name())
|
||||
skillFile := filepath.Join(skillPath, SkillFileName)
|
||||
|
||||
// Check if SKILL.md exists
|
||||
if _, err := os.Stat(skillFile); os.IsNotExist(err) {
|
||||
continue
|
||||
}
|
||||
|
||||
// Read and parse SKILL.md
|
||||
content, err := os.ReadFile(skillFile)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
skill, err := ParseSkillFile(string(content))
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
// Set filesystem paths
|
||||
skill.BasePath = skillPath
|
||||
skill.FilePath = skillFile
|
||||
|
||||
// Cache the skill
|
||||
l.discoveredSkills[skill.Name] = skill
|
||||
|
||||
metadata = append(metadata, skill.ToMetadata())
|
||||
}
|
||||
|
||||
return metadata, nil
|
||||
}
|
||||
|
||||
// LoadSkillInstructions loads the full instructions of a skill (Level 2)
|
||||
// Returns the cached skill if already loaded
|
||||
func (l *Loader) LoadSkillInstructions(skillName string) (*Skill, error) {
|
||||
// Check cache first
|
||||
if skill, ok := l.discoveredSkills[skillName]; ok {
|
||||
if skill.Loaded {
|
||||
return skill, nil
|
||||
}
|
||||
}
|
||||
|
||||
// Search for the skill in all directories
|
||||
for _, dir := range l.skillDirs {
|
||||
skill, err := l.loadSkillFromDirectory(dir, skillName)
|
||||
if err == nil {
|
||||
l.discoveredSkills[skillName] = skill
|
||||
return skill, nil
|
||||
}
|
||||
}
|
||||
|
||||
return nil, fmt.Errorf("skill not found: %s", skillName)
|
||||
}
|
||||
|
||||
// loadSkillFromDirectory attempts to load a skill from a specific directory
|
||||
func (l *Loader) loadSkillFromDirectory(dir, skillName string) (*Skill, error) {
|
||||
// First, check if we can find by directory name matching skill name
|
||||
skillPath := filepath.Join(dir, skillName)
|
||||
skillFile := filepath.Join(skillPath, SkillFileName)
|
||||
|
||||
if _, err := os.Stat(skillFile); err == nil {
|
||||
return l.loadSkillFile(skillPath, skillFile)
|
||||
}
|
||||
|
||||
// Otherwise, scan all subdirectories to find the skill by name
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
for _, entry := range entries {
|
||||
if !entry.IsDir() {
|
||||
continue
|
||||
}
|
||||
|
||||
skillPath := filepath.Join(dir, entry.Name())
|
||||
skillFile := filepath.Join(skillPath, SkillFileName)
|
||||
|
||||
if _, err := os.Stat(skillFile); os.IsNotExist(err) {
|
||||
continue
|
||||
}
|
||||
|
||||
content, err := os.ReadFile(skillFile)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
skill, err := ParseSkillFile(string(content))
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
if skill.Name == skillName {
|
||||
skill.BasePath = skillPath
|
||||
skill.FilePath = skillFile
|
||||
return skill, nil
|
||||
}
|
||||
}
|
||||
|
||||
return nil, fmt.Errorf("skill not found in %s: %s", dir, skillName)
|
||||
}
|
||||
|
||||
// loadSkillFile reads and parses a SKILL.md file
|
||||
func (l *Loader) loadSkillFile(basePath, filePath string) (*Skill, error) {
|
||||
content, err := os.ReadFile(filePath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read skill file: %w", err)
|
||||
}
|
||||
|
||||
skill, err := ParseSkillFile(string(content))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
skill.BasePath = basePath
|
||||
skill.FilePath = filePath
|
||||
|
||||
return skill, nil
|
||||
}
|
||||
|
||||
// LoadSkillFile loads an additional file from a skill directory (Level 3)
|
||||
// The filePath should be relative to the skill's base directory
|
||||
func (l *Loader) LoadSkillFile(skillName, relativePath string) (*SkillFile, error) {
|
||||
// Get the skill first
|
||||
skill, ok := l.discoveredSkills[skillName]
|
||||
if !ok {
|
||||
// Try to load the skill
|
||||
var err error
|
||||
skill, err = l.LoadSkillInstructions(skillName)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("skill not found: %s", skillName)
|
||||
}
|
||||
}
|
||||
|
||||
// Validate and resolve the file path
|
||||
cleanPath := filepath.Clean(relativePath)
|
||||
|
||||
// Security: prevent path traversal
|
||||
if strings.HasPrefix(cleanPath, "..") || filepath.IsAbs(cleanPath) {
|
||||
return nil, fmt.Errorf("invalid file path: %s", relativePath)
|
||||
}
|
||||
|
||||
fullPath := filepath.Join(skill.BasePath, cleanPath)
|
||||
|
||||
// Verify the file is within the skill directory
|
||||
absSkillPath, err := filepath.Abs(skill.BasePath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
absFilePath, err := filepath.Abs(fullPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !strings.HasPrefix(absFilePath, absSkillPath) {
|
||||
return nil, fmt.Errorf("file path outside skill directory: %s", relativePath)
|
||||
}
|
||||
|
||||
// Read the file
|
||||
content, err := os.ReadFile(fullPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read file: %w", err)
|
||||
}
|
||||
|
||||
return &SkillFile{
|
||||
Name: relativePath,
|
||||
Path: absFilePath, // Use absolute path for sandbox execution
|
||||
Content: string(content),
|
||||
IsScript: IsScript(relativePath),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// ListSkillFiles lists all files in a skill directory
|
||||
func (l *Loader) ListSkillFiles(skillName string) ([]string, error) {
|
||||
skill, ok := l.discoveredSkills[skillName]
|
||||
if !ok {
|
||||
var err error
|
||||
skill, err = l.LoadSkillInstructions(skillName)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("skill not found: %s", skillName)
|
||||
}
|
||||
}
|
||||
|
||||
var files []string
|
||||
|
||||
err := filepath.Walk(skill.BasePath, func(path string, info os.FileInfo, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if info.IsDir() {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Get relative path
|
||||
relPath, err := filepath.Rel(skill.BasePath, path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
files = append(files, relPath)
|
||||
return nil
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to list skill files: %w", err)
|
||||
}
|
||||
|
||||
return files, nil
|
||||
}
|
||||
|
||||
// GetSkillByName returns a cached skill by name
|
||||
func (l *Loader) GetSkillByName(name string) (*Skill, bool) {
|
||||
skill, ok := l.discoveredSkills[name]
|
||||
return skill, ok
|
||||
}
|
||||
|
||||
// GetSkillBasePath returns the base path for a skill (always absolute)
|
||||
func (l *Loader) GetSkillBasePath(skillName string) (string, error) {
|
||||
skill, ok := l.discoveredSkills[skillName]
|
||||
if !ok {
|
||||
var err error
|
||||
skill, err = l.LoadSkillInstructions(skillName)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("skill not found: %s", skillName)
|
||||
}
|
||||
}
|
||||
// Return absolute path for consistent sandbox execution
|
||||
return filepath.Abs(skill.BasePath)
|
||||
}
|
||||
|
||||
// Reload clears the cache and rediscovers all skills
|
||||
func (l *Loader) Reload() ([]*SkillMetadata, error) {
|
||||
l.discoveredSkills = make(map[string]*Skill)
|
||||
return l.DiscoverSkills()
|
||||
}
|
||||
@@ -0,0 +1,601 @@
|
||||
package skills
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/logger"
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
)
|
||||
|
||||
// artifactOutputEnvVar is the name of the environment variable that WeKnora
|
||||
// injects into every skill script execution. The value points to the
|
||||
// convention-driven directory where the script should drop artifacts the user
|
||||
// will be able to download after the turn completes.
|
||||
//
|
||||
// The name is stable across releases; skills reference it via os.getenv(...)
|
||||
// so they never hard-code the path.
|
||||
const artifactOutputEnvVar = "WEKNORA_SKILL_OUTPUT_DIR"
|
||||
|
||||
// sessionInputEnvVar points skill scripts at user-uploaded files restored into
|
||||
// the current session's Cube. Inputs are separate from generated artifacts.
|
||||
const sessionInputEnvVar = "WEKNORA_SESSION_INPUT_DIR"
|
||||
|
||||
// artifactHistoryEnvVar is the name of the environment variable that points
|
||||
// to the root artifact output directory (/workspace/output). Skill scripts
|
||||
// can use this to self-discover artifacts from prior runs when they need to
|
||||
// chain without LLM mediation.
|
||||
const artifactHistoryEnvVar = "WEKNORA_SKILL_HISTORY_ROOT"
|
||||
|
||||
// skillDirEnvVar points a script at its own directory inside the sandbox
|
||||
// image. Installed skills run with /workspace as WorkDir, so this is how a
|
||||
// script reaches the data and helpers that were installed beside it. The
|
||||
// install-time verification pass exports the same name.
|
||||
const skillDirEnvVar = "WEKNORA_SKILL_DIR"
|
||||
|
||||
// pythonPathEnvVar / nodePathEnvVar carry the per-session extra-packages
|
||||
// overlay (see sandbox.SessionSkillPackageDir). They are injected rather than
|
||||
// left for the skill to declare so a stored PYTHONPATH cannot displace them.
|
||||
const pythonPathEnvVar = "PYTHONPATH"
|
||||
const nodePathEnvVar = "NODE_PATH"
|
||||
|
||||
// InjectedSandboxEnvVars is every name ExecuteScript writes into the sandbox
|
||||
// environment. The skill-env declaration blacklist must reject these so a
|
||||
// stored value cannot redirect artifacts, the skill directory, or the session
|
||||
// input tree. Credential names such as WEKNORA_API_KEY are not in this list.
|
||||
func InjectedSandboxEnvVars() []string {
|
||||
return []string{
|
||||
artifactOutputEnvVar,
|
||||
sessionInputEnvVar,
|
||||
artifactHistoryEnvVar,
|
||||
skillDirEnvVar,
|
||||
pythonPathEnvVar,
|
||||
nodePathEnvVar,
|
||||
}
|
||||
}
|
||||
|
||||
// defaultArtifactOutputDir is used when neither the environment variable
|
||||
// (WEKNORA_SKILL_OUTPUT_DIR) nor the ExecuteConfig.Env has an override.
|
||||
// /workspace/output sits inside the base sandbox image's writable tree and
|
||||
// is guaranteed to survive across Execute calls for the same session (Cube
|
||||
// SessionBoundManager keeps the MicroVM alive between calls).
|
||||
const defaultArtifactOutputDir = "/workspace/output"
|
||||
|
||||
// ArtifactOutputDir returns the absolute path (inside the sandbox) where
|
||||
// skill scripts should write artifacts for this turn. It is exported so
|
||||
// callers such as ArtifactCollector can list the same directory when
|
||||
// draining artifacts after Execute returns.
|
||||
//
|
||||
// Resolution order (first usable wins):
|
||||
// 1. WEKNORA_SKILL_OUTPUT_DIR from the host environment (ops override), when
|
||||
// it names a directory inside the session workspace.
|
||||
// 2. defaultArtifactOutputDir.
|
||||
//
|
||||
// The override goes through sandbox.ValidatedSessionOutputDir, the same gate
|
||||
// the sandbox applies to a tenant's override. An operator who points this
|
||||
// outside /workspace would otherwise send the readers (this function feeds the
|
||||
// sandbox file tools and ArtifactCollector) to a directory no skill can write,
|
||||
// since execution refuses the same path and falls back.
|
||||
//
|
||||
// Callers are expected to treat the returned string as read-only: the path
|
||||
// is normalised (no trailing slash) so it can be joined safely.
|
||||
func ArtifactOutputDir() string {
|
||||
if v := strings.TrimSpace(os.Getenv(artifactOutputEnvVar)); v != "" {
|
||||
if clean, ok := sandbox.ValidatedSessionOutputDir(v); ok {
|
||||
return clean
|
||||
}
|
||||
}
|
||||
return defaultArtifactOutputDir
|
||||
}
|
||||
|
||||
// SkillOutputDir returns the artifact output directory for skill executions.
|
||||
// All skills write to the same root directory (/workspace/output/) to enable
|
||||
// collaboration and file sharing between different skill executions.
|
||||
func (m *Manager) SkillOutputDir(sessionID, skillName string) string {
|
||||
return ArtifactOutputDir()
|
||||
}
|
||||
|
||||
// Manager manages skills lifecycle including discovery, loading, and script execution
|
||||
// It coordinates between the Loader (filesystem operations) and Sandbox (script execution)
|
||||
type Manager struct {
|
||||
loader *Loader
|
||||
sandboxMgr sandbox.Manager
|
||||
|
||||
// tenantSource holds the skills installed into this run's sandbox image.
|
||||
// When set it is the only source the model is told about: the host
|
||||
// skills/preloaded directory is not what execute_skill_script would find
|
||||
// inside the sandbox.
|
||||
tenantSource SkillSource
|
||||
|
||||
// envResolver supplies the per-caller environment for one execution. It
|
||||
// is nil when the run has no installed skills, in which case execution
|
||||
// keeps exactly its previous behaviour.
|
||||
envResolver SkillEnvResolver
|
||||
|
||||
// Configuration
|
||||
skillDirs []string
|
||||
allowedSkills []string // Empty means all skills are allowed
|
||||
enabled bool
|
||||
|
||||
// Cache
|
||||
metadataCache []*SkillMetadata
|
||||
mu sync.RWMutex
|
||||
}
|
||||
|
||||
// ManagerConfig holds configuration for the skill manager
|
||||
type ManagerConfig struct {
|
||||
SkillDirs []string // Directories to search for skills
|
||||
AllowedSkills []string // Skill names whitelist (empty = allow all)
|
||||
Enabled bool // Whether skills are enabled
|
||||
}
|
||||
|
||||
// NewManager creates a new skill manager with the given configuration
|
||||
func NewManager(config *ManagerConfig, sandboxMgr sandbox.Manager) *Manager {
|
||||
if config == nil {
|
||||
config = &ManagerConfig{
|
||||
Enabled: false,
|
||||
}
|
||||
}
|
||||
|
||||
return &Manager{
|
||||
loader: NewLoader(config.SkillDirs),
|
||||
sandboxMgr: sandboxMgr,
|
||||
skillDirs: config.SkillDirs,
|
||||
allowedSkills: config.AllowedSkills,
|
||||
enabled: config.Enabled,
|
||||
}
|
||||
}
|
||||
|
||||
// IsEnabled returns whether skills are enabled
|
||||
func (m *Manager) IsEnabled() bool {
|
||||
return m.enabled
|
||||
}
|
||||
|
||||
// WithTenantSource attaches the skills an administrator installed into the
|
||||
// sandbox config this run booted from. It is part of construction - callers
|
||||
// must invoke it before Initialize, i.e. before the engine can reach the
|
||||
// manager - so it takes no lock.
|
||||
func (m *Manager) WithTenantSource(source SkillSource) *Manager {
|
||||
m.tenantSource = source
|
||||
return m
|
||||
}
|
||||
|
||||
// WithEnvResolver attaches the per-caller environment resolver. Like
|
||||
// WithTenantSource it is part of construction and must be invoked before
|
||||
// Initialize, so it takes no lock.
|
||||
func (m *Manager) WithEnvResolver(resolver SkillEnvResolver) *Manager {
|
||||
m.envResolver = resolver
|
||||
return m
|
||||
}
|
||||
|
||||
// resolveSource decides which source owns one skill name. An installed image
|
||||
// is the only copy the sandbox can run: falling back to a host preloaded
|
||||
// skill would advertise files that are not in the image.
|
||||
func (m *Manager) resolveSource(skillName string) SkillSource {
|
||||
if m.tenantSource != nil {
|
||||
return m.tenantSource
|
||||
}
|
||||
return m.loader
|
||||
}
|
||||
|
||||
// discoverAllSkills returns the set the model is told about. When skills are
|
||||
// installed into the sandbox image, that image is the source of truth; the
|
||||
// deployment's skills/preloaded directory is not what execute_skill_script
|
||||
// would find inside the sandbox.
|
||||
func (m *Manager) discoverAllSkills() ([]*SkillMetadata, error) {
|
||||
if m.tenantSource != nil {
|
||||
return m.tenantSource.DiscoverSkills()
|
||||
}
|
||||
return m.loader.Reload()
|
||||
}
|
||||
|
||||
// Initialize discovers all skills and caches their metadata
|
||||
// This should be called at startup
|
||||
func (m *Manager) Initialize(ctx context.Context) error {
|
||||
if !m.enabled {
|
||||
return nil
|
||||
}
|
||||
|
||||
metadata, err := m.discoverAllSkills()
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to discover skills: %w", err)
|
||||
}
|
||||
|
||||
// Filter by allowed skills if specified
|
||||
if len(m.allowedSkills) > 0 {
|
||||
metadata = m.filterAllowedSkills(metadata)
|
||||
}
|
||||
|
||||
m.mu.Lock()
|
||||
m.metadataCache = metadata
|
||||
m.mu.Unlock()
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// filterAllowedSkills filters metadata to only include allowed skills
|
||||
func (m *Manager) filterAllowedSkills(metadata []*SkillMetadata) []*SkillMetadata {
|
||||
if len(m.allowedSkills) == 0 {
|
||||
return metadata
|
||||
}
|
||||
|
||||
allowedSet := make(map[string]bool)
|
||||
for _, name := range m.allowedSkills {
|
||||
allowedSet[name] = true
|
||||
}
|
||||
|
||||
var filtered []*SkillMetadata
|
||||
for _, meta := range metadata {
|
||||
if allowedSet[meta.Name] {
|
||||
filtered = append(filtered, meta)
|
||||
}
|
||||
}
|
||||
return filtered
|
||||
}
|
||||
|
||||
// GetAllMetadata returns metadata for all discovered skills
|
||||
// This is used for system prompt injection (Level 1)
|
||||
func (m *Manager) GetAllMetadata() []*SkillMetadata {
|
||||
if !m.enabled {
|
||||
return nil
|
||||
}
|
||||
|
||||
m.mu.RLock()
|
||||
defer m.mu.RUnlock()
|
||||
|
||||
// Return a copy to prevent external modification
|
||||
result := make([]*SkillMetadata, len(m.metadataCache))
|
||||
copy(result, m.metadataCache)
|
||||
return result
|
||||
}
|
||||
|
||||
// LoadSkill loads the full instructions of a skill (Level 2)
|
||||
func (m *Manager) LoadSkill(ctx context.Context, skillName string) (*Skill, error) {
|
||||
if !m.enabled {
|
||||
return nil, fmt.Errorf("skills are not enabled")
|
||||
}
|
||||
|
||||
// Check if skill is allowed
|
||||
if !m.isSkillAllowed(skillName) {
|
||||
return nil, fmt.Errorf("skill not allowed: %s", skillName)
|
||||
}
|
||||
|
||||
return m.resolveSource(skillName).LoadSkillInstructions(skillName)
|
||||
}
|
||||
|
||||
// isSkillAllowed checks if a skill is in the allowed list
|
||||
func (m *Manager) isSkillAllowed(skillName string) bool {
|
||||
if len(m.allowedSkills) == 0 {
|
||||
return true
|
||||
}
|
||||
for _, name := range m.allowedSkills {
|
||||
if name == skillName {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ReadSkillFile reads an additional file from a skill directory (Level 3)
|
||||
func (m *Manager) ReadSkillFile(ctx context.Context, skillName, filePath string) (string, error) {
|
||||
if !m.enabled {
|
||||
return "", fmt.Errorf("skills are not enabled")
|
||||
}
|
||||
|
||||
if !m.isSkillAllowed(skillName) {
|
||||
return "", fmt.Errorf("skill not allowed: %s", skillName)
|
||||
}
|
||||
|
||||
file, err := m.resolveSource(skillName).LoadSkillFile(skillName, filePath)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
return file.Content, nil
|
||||
}
|
||||
|
||||
// ListSkillFiles lists all files in a skill directory
|
||||
func (m *Manager) ListSkillFiles(ctx context.Context, skillName string) ([]string, error) {
|
||||
if !m.enabled {
|
||||
return nil, fmt.Errorf("skills are not enabled")
|
||||
}
|
||||
|
||||
if !m.isSkillAllowed(skillName) {
|
||||
return nil, fmt.Errorf("skill not allowed: %s", skillName)
|
||||
}
|
||||
|
||||
return m.resolveSource(skillName).ListSkillFiles(skillName)
|
||||
}
|
||||
|
||||
// SandboxSkillDir reports where a skill lives inside the sandbox image, and
|
||||
// whether that path means anything to say out loud.
|
||||
//
|
||||
// Only an installed skill has one. A preloaded skill is uploaded from the host
|
||||
// for the duration of a single call, so its base path names a directory on the
|
||||
// WeKnora machine that no shell command in the sandbox can reach — telling the
|
||||
// model about it would be worse than saying nothing.
|
||||
func (m *Manager) SandboxSkillDir(skillName string) (string, bool) {
|
||||
if m == nil || !m.enabled || !m.isSkillAllowed(skillName) {
|
||||
return "", false
|
||||
}
|
||||
image, ok := m.resolveSource(skillName).(imageSkillSource)
|
||||
if !ok {
|
||||
return "", false
|
||||
}
|
||||
dir, err := image.GetSkillBasePath(skillName)
|
||||
if err != nil {
|
||||
return "", false
|
||||
}
|
||||
dir = strings.TrimSpace(dir)
|
||||
return dir, dir != ""
|
||||
}
|
||||
|
||||
// ExecuteScript executes a script from a skill in the sandbox
|
||||
func (m *Manager) ExecuteScript(ctx context.Context, skillName, scriptPath string, args []string, stdin string) (*sandbox.ExecuteResult, error) {
|
||||
if !m.enabled {
|
||||
return nil, fmt.Errorf("skills are not enabled")
|
||||
}
|
||||
|
||||
if !m.isSkillAllowed(skillName) {
|
||||
return nil, fmt.Errorf("skill not allowed: %s", skillName)
|
||||
}
|
||||
|
||||
// Verify sandbox manager is available
|
||||
if m.sandboxMgr == nil {
|
||||
return nil, fmt.Errorf("sandbox is not configured")
|
||||
}
|
||||
|
||||
source := m.resolveSource(skillName)
|
||||
|
||||
// Get the skill base path
|
||||
basePath, err := source.GetSkillBasePath(skillName)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Prepare execution config
|
||||
logger.Info(ctx, "[Tool][ExecuteScript]:Prepare execution config")
|
||||
sessionID, _ := types.SessionIDFromContext(ctx)
|
||||
|
||||
// Compute the artifact output directory. All skills share the same root
|
||||
// directory (/workspace/output/) to enable collaboration and file sharing
|
||||
// between different skill executions in the same session.
|
||||
// Skill scripts read the directory via WEKNORA_SKILL_OUTPUT_DIR; the
|
||||
// root (for cross-run discovery) is available via WEKNORA_SKILL_HISTORY_ROOT.
|
||||
outputDir := m.SkillOutputDir(sessionID, skillName)
|
||||
env := map[string]string{
|
||||
artifactOutputEnvVar: outputDir,
|
||||
artifactHistoryEnvVar: ArtifactOutputDir(),
|
||||
}
|
||||
applySessionPackagePath(env, skillName)
|
||||
// SessionFileStore advertises the "sandbox provides per-session file
|
||||
// storage" capability. When present we can safely expose the input
|
||||
// staging directory and pre-materialise the output directory; when
|
||||
// absent, directories are materialised during script execution.
|
||||
fileStore := sessionFileStoreFromManager(m.sandboxMgr)
|
||||
if fileStore != nil {
|
||||
env[sessionInputEnvVar] = sandbox.SessionInputRoot
|
||||
if sessionID != "" {
|
||||
if err := fileStore.EnsureSessionDir(ctx, sessionID, outputDir); err != nil {
|
||||
logger.Warnf(ctx, "[Tool][ExecuteScript] pre-create output dir %s failed: %v", outputDir, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Per-caller values are resolved here rather than baked into the sandbox
|
||||
// at creation: an IM thread can have several people sharing one sandbox,
|
||||
// so each turn's values must belong to that turn's speaker and must not
|
||||
// linger where the next person could read them with `env`.
|
||||
//
|
||||
// The resolver runs after the artifact keys above are seeded, so
|
||||
// ApplyResolvedEnv's skip-existing rule protects WEKNORA_SKILL_OUTPUT_DIR
|
||||
// and WEKNORA_SKILL_HISTORY_ROOT (always) plus WEKNORA_SESSION_INPUT_DIR
|
||||
// (when a session file store exists). skillDirEnvVar is set later inside
|
||||
// buildSkillExecuteConfig by an unconditional write. The write-time
|
||||
// blacklist of skills.InjectedSandboxEnvVars (service.validateUserEnvName)
|
||||
// is what covers a name that was not seeded on this path. Other WEKNORA_*
|
||||
// names (API keys, base URLs) are ordinary credentials a skill may declare.
|
||||
if m.envResolver != nil {
|
||||
resolved, missing, err := m.envResolver.ResolveEnv(ctx, skillName)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(missing) > 0 {
|
||||
return nil, &MissingSkillEnvError{SkillName: skillName, Names: missing}
|
||||
}
|
||||
ApplyResolvedEnv(env, resolved)
|
||||
}
|
||||
|
||||
config, err := buildSkillExecuteConfig(
|
||||
source, skillName, scriptPath, basePath, args, stdin, env, sessionID,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Execute in sandbox
|
||||
return m.sandboxMgr.Execute(ctx, config)
|
||||
}
|
||||
|
||||
// buildSkillExecuteConfig turns one skill script into an execution request.
|
||||
//
|
||||
// Three cases:
|
||||
// - A session /workspace file (from write_sandbox_file): run in place with
|
||||
// the installed skill's interpreter. Preloaded skills cannot do this.
|
||||
// - An installed image skill: run the relative path in place; no host upload.
|
||||
// - A preloaded skill: upload from the host and run with the skill directory
|
||||
// as WorkDir.
|
||||
func buildSkillExecuteConfig(
|
||||
source SkillSource,
|
||||
skillName, scriptPath, basePath string,
|
||||
args []string,
|
||||
stdin string,
|
||||
env map[string]string,
|
||||
sessionID string,
|
||||
) (*sandbox.ExecuteConfig, error) {
|
||||
if workspace, ok := sandbox.RunnableWorkspaceScript(scriptPath); ok {
|
||||
return workspaceSkillExecuteConfig(source, skillName, workspace, basePath, args, stdin, env, sessionID)
|
||||
}
|
||||
|
||||
image, installed := source.(imageSkillSource)
|
||||
if !installed {
|
||||
// Load the script file to verify it exists and is a script
|
||||
file, err := source.LoadSkillFile(skillName, scriptPath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to load script: %w", err)
|
||||
}
|
||||
if !file.IsScript {
|
||||
return nil, fmt.Errorf("file is not an executable script: %s", scriptPath)
|
||||
}
|
||||
return &sandbox.ExecuteConfig{
|
||||
Script: file.Path,
|
||||
Args: args,
|
||||
WorkDir: basePath,
|
||||
Stdin: stdin,
|
||||
Env: env,
|
||||
SessionID: sessionID,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// The archive is deliberately not consulted: the image is what executes,
|
||||
// and a skill whose archive failed to store is still installed and
|
||||
// runnable. That leaves the extension as the only check available here,
|
||||
// which is also the one the executor's interpreter choice depends on.
|
||||
if !IsScript(scriptPath) {
|
||||
return nil, fmt.Errorf("file is not an executable script: %s", scriptPath)
|
||||
}
|
||||
remoteScript, err := image.RemoteScriptPath(skillName, scriptPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// The install-time verification pass exports the skill directory under
|
||||
// this name, so the environment a script is checked in is the environment
|
||||
// it is later called in.
|
||||
env[skillDirEnvVar] = basePath
|
||||
return &sandbox.ExecuteConfig{
|
||||
RemoteScriptPath: remoteScript,
|
||||
Args: args,
|
||||
Stdin: stdin,
|
||||
Env: env,
|
||||
SessionID: sessionID,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// workspaceSkillExecuteConfig runs a session-written /workspace file with the
|
||||
// installed skill's interpreter. Preloaded (host-uploaded) skills have no
|
||||
// in-sandbox venv to attach, so those calls are rejected with a shell_exec hint.
|
||||
func workspaceSkillExecuteConfig(
|
||||
source SkillSource,
|
||||
skillName, workspacePath, basePath string,
|
||||
args []string,
|
||||
stdin string,
|
||||
env map[string]string,
|
||||
sessionID string,
|
||||
) (*sandbox.ExecuteConfig, error) {
|
||||
if !IsScript(workspacePath) {
|
||||
return nil, fmt.Errorf("file is not an executable script: %s", workspacePath)
|
||||
}
|
||||
if _, installed := source.(imageSkillSource); !installed {
|
||||
return nil, fmt.Errorf(
|
||||
"script_path %q is a session workspace file; this skill is not installed in the sandbox image, so execute_skill_script cannot attach its environment. Run it with shell_exec, or pass a skill-relative path such as scripts/foo.py",
|
||||
workspacePath,
|
||||
)
|
||||
}
|
||||
skillDir, ok := sandbox.ValidatedImageSkillDir(basePath)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("cannot run workspace script %q for skill %q: no installed skill directory", workspacePath, skillName)
|
||||
}
|
||||
env[skillDirEnvVar] = skillDir
|
||||
return &sandbox.ExecuteConfig{
|
||||
RemoteScriptPath: workspacePath,
|
||||
SkillDir: skillDir,
|
||||
Args: args,
|
||||
Stdin: stdin,
|
||||
Env: env,
|
||||
SessionID: sessionID,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// sessionFileStoreFromManager returns the sandbox manager's effective
|
||||
// session filesystem capability, or nil when the backend cannot expose one.
|
||||
// Isolated in a helper so callers stay free of provider-specific branches.
|
||||
func sessionFileStoreFromManager(mgr sandbox.Manager) sandbox.SessionFileStore {
|
||||
provider, ok := mgr.(sandbox.SessionCapabilityProvider)
|
||||
if !ok || provider == nil {
|
||||
return nil
|
||||
}
|
||||
return provider.SessionFileStore()
|
||||
}
|
||||
|
||||
// GetSkillInfo returns detailed information about a skill
|
||||
func (m *Manager) GetSkillInfo(ctx context.Context, skillName string) (*SkillInfo, error) {
|
||||
if !m.enabled {
|
||||
return nil, fmt.Errorf("skills are not enabled")
|
||||
}
|
||||
|
||||
if !m.isSkillAllowed(skillName) {
|
||||
return nil, fmt.Errorf("skill not allowed: %s", skillName)
|
||||
}
|
||||
|
||||
source := m.resolveSource(skillName)
|
||||
skill, err := source.LoadSkillInstructions(skillName)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
files, err := source.ListSkillFiles(skillName)
|
||||
if err != nil {
|
||||
files = []string{} // Non-fatal error
|
||||
}
|
||||
|
||||
return &SkillInfo{
|
||||
Name: skill.Name,
|
||||
Description: skill.Description,
|
||||
BasePath: skill.BasePath,
|
||||
Instructions: skill.Instructions,
|
||||
Files: files,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// SkillInfo provides detailed information about a skill
|
||||
type SkillInfo struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
BasePath string `json:"base_path"`
|
||||
Instructions string `json:"instructions"`
|
||||
Files []string `json:"files"`
|
||||
}
|
||||
|
||||
// Reload refreshes the skill cache by rediscovering all skills
|
||||
func (m *Manager) Reload(ctx context.Context) error {
|
||||
if !m.enabled {
|
||||
return nil
|
||||
}
|
||||
|
||||
metadata, err := m.discoverAllSkills()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if len(m.allowedSkills) > 0 {
|
||||
metadata = m.filterAllowedSkills(metadata)
|
||||
}
|
||||
|
||||
m.mu.Lock()
|
||||
m.metadataCache = metadata
|
||||
m.mu.Unlock()
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// Cleanup releases resources
|
||||
func (m *Manager) Cleanup(ctx context.Context) error {
|
||||
if m.sandboxMgr != nil {
|
||||
return m.sandboxMgr.Cleanup(ctx)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,292 @@
|
||||
// Package skills provides Agent Skills functionality following Claude's Progressive Disclosure pattern.
|
||||
// Skills are modular capabilities that extend the agent's functionality through instruction files.
|
||||
package skills
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"errors"
|
||||
"fmt"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
// Skill validation constants following Claude's specification
|
||||
const (
|
||||
MaxNameLength = 64
|
||||
MaxDescriptionLength = 1024
|
||||
SkillFileName = "SKILL.md"
|
||||
)
|
||||
|
||||
// Reserved words that cannot be used in skill names
|
||||
var reservedWords = []string{"anthropic", "claude"}
|
||||
|
||||
// namePattern is the install identity: a single path segment of letters
|
||||
// (any script), digits, hyphens, and underscores. Display titles such as
|
||||
// "Word / DOCX" are not this; those are rewritten via slug / slugify
|
||||
// before Validate. Slashes stay out so the name cannot walk SkillsImageRoot.
|
||||
var namePattern = regexp.MustCompile(`^[\p{L}\p{N}_-]+$`)
|
||||
|
||||
var skillNameSepRE = regexp.MustCompile(`[^\p{L}\p{N}]+`)
|
||||
|
||||
// xmlTagPattern detects XML tags in content
|
||||
var xmlTagPattern = regexp.MustCompile(`<[^>]+>`)
|
||||
|
||||
// Skill represents a loaded skill with its metadata and content
|
||||
// It follows the Progressive Disclosure pattern:
|
||||
// - Level 1 (Metadata): Name and Description are always loaded
|
||||
// - Level 2 (Instructions): The main body of SKILL.md, loaded on demand
|
||||
// - Level 3 (Resources): Additional files in the skill directory, loaded as needed
|
||||
type Skill struct {
|
||||
// Metadata (Level 1) - always loaded
|
||||
Name string `yaml:"name"`
|
||||
Description string `yaml:"description"`
|
||||
// Slug is an optional filesystem-safe id. ClawHub / SkillHub often put a
|
||||
// display title in name ("Word / DOCX") and the install id in slug
|
||||
// ("word-docx"). When name is not a valid install identity, slug wins.
|
||||
Slug string `yaml:"slug,omitempty"`
|
||||
|
||||
// Filesystem information
|
||||
BasePath string // Absolute path to the skill directory
|
||||
FilePath string // Absolute path to SKILL.md
|
||||
|
||||
// Instructions (Level 2) - loaded on demand
|
||||
Instructions string // The main body of SKILL.md (after frontmatter)
|
||||
Loaded bool // Whether Level 2 instructions have been loaded
|
||||
|
||||
// FrontmatterRepaired is true when the YAML between the --- markers had
|
||||
// to be repaired (keys nested under a scalar, or an unquoted colon)
|
||||
// before it would parse. The original SKILL.md is unchanged; callers
|
||||
// that install the archive should tell the user so they can fix it.
|
||||
FrontmatterRepaired bool
|
||||
}
|
||||
|
||||
// SkillMetadata represents the minimal metadata for system prompt injection (Level 1)
|
||||
// This is the lightweight representation used during skill discovery
|
||||
type SkillMetadata struct {
|
||||
Name string
|
||||
Description string
|
||||
BasePath string // Path to skill directory for later loading
|
||||
}
|
||||
|
||||
// SkillFile represents an additional file within a skill directory (Level 3)
|
||||
type SkillFile struct {
|
||||
Name string // Filename (e.g., "FORMS.md", "scripts/validate.py")
|
||||
Path string // Absolute path to the file
|
||||
Content string // File content
|
||||
IsScript bool // Whether this is an executable script
|
||||
}
|
||||
|
||||
// Validate checks if the skill metadata is valid according to Claude's specification
|
||||
func (s *Skill) Validate() error {
|
||||
// Validate name
|
||||
if s.Name == "" {
|
||||
return errors.New("skill name is required")
|
||||
}
|
||||
if n := utf8.RuneCountInString(s.Name); n > MaxNameLength {
|
||||
return fmt.Errorf("skill name is %d characters; maximum is %d", n, MaxNameLength)
|
||||
}
|
||||
if !namePattern.MatchString(s.Name) {
|
||||
return errors.New("skill name must contain only letters, numbers, hyphens, and underscores")
|
||||
}
|
||||
for _, reserved := range reservedWords {
|
||||
if strings.Contains(s.Name, reserved) {
|
||||
return fmt.Errorf("skill name cannot contain reserved word: %s", reserved)
|
||||
}
|
||||
}
|
||||
if xmlTagPattern.MatchString(s.Name) {
|
||||
return errors.New("skill name cannot contain XML tags")
|
||||
}
|
||||
|
||||
// Validate description
|
||||
if s.Description == "" {
|
||||
return errors.New("skill description is required")
|
||||
}
|
||||
if n := utf8.RuneCountInString(s.Description); n > MaxDescriptionLength {
|
||||
return fmt.Errorf("skill description is %d characters; maximum is %d", n, MaxDescriptionLength)
|
||||
}
|
||||
if xmlTagPattern.MatchString(s.Description) {
|
||||
return errors.New("skill description cannot contain XML tags")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// applyInstallName picks the directory / tool identity. Third-party SKILL.md
|
||||
// files often put a display title in name ("Word / DOCX") and a kebab-case
|
||||
// id in slug ("word-docx"). A title that is already a valid identity is
|
||||
// left alone so Chinese names such as 律师助手 stay as-is.
|
||||
func (s *Skill) applyInstallName() error {
|
||||
if s == nil {
|
||||
return errors.New("skill name is required")
|
||||
}
|
||||
if installableSkillName(s.Name) {
|
||||
s.Name = strings.TrimSpace(s.Name)
|
||||
return nil
|
||||
}
|
||||
if installableSkillName(s.Slug) {
|
||||
s.Name = strings.TrimSpace(s.Slug)
|
||||
return nil
|
||||
}
|
||||
if derived := slugifySkillName(s.Name); installableSkillName(derived) {
|
||||
s.Name = derived
|
||||
return nil
|
||||
}
|
||||
return errors.New("skill name must contain only letters, numbers, hyphens, and underscores (or set slug)")
|
||||
}
|
||||
|
||||
func installableSkillName(name string) bool {
|
||||
name = strings.TrimSpace(name)
|
||||
return name != "" && namePattern.MatchString(name)
|
||||
}
|
||||
|
||||
func slugifySkillName(name string) string {
|
||||
s := strings.ToLower(strings.TrimSpace(name))
|
||||
s = skillNameSepRE.ReplaceAllString(s, "-")
|
||||
return strings.Trim(s, "-_")
|
||||
}
|
||||
|
||||
// ToMetadata converts a Skill to its lightweight metadata representation
|
||||
func (s *Skill) ToMetadata() *SkillMetadata {
|
||||
return &SkillMetadata{
|
||||
Name: s.Name,
|
||||
Description: s.Description,
|
||||
BasePath: s.BasePath,
|
||||
}
|
||||
}
|
||||
|
||||
// ParseSkillFile parses a SKILL.md file content and extracts metadata and body
|
||||
// It handles YAML frontmatter enclosed in --- delimiters
|
||||
func ParseSkillFile(content string) (*Skill, error) {
|
||||
skill := &Skill{}
|
||||
|
||||
// Check for YAML frontmatter
|
||||
if !strings.HasPrefix(strings.TrimSpace(content), "---") {
|
||||
return nil, errors.New("SKILL.md must start with YAML frontmatter (---)")
|
||||
}
|
||||
|
||||
// Find the end of frontmatter
|
||||
scanner := bufio.NewScanner(strings.NewReader(content))
|
||||
var frontmatterLines []string
|
||||
var bodyLines []string
|
||||
inFrontmatter := false
|
||||
frontmatterEnded := false
|
||||
|
||||
for scanner.Scan() {
|
||||
line := scanner.Text()
|
||||
|
||||
if !inFrontmatter && !frontmatterEnded && strings.TrimSpace(line) == "---" {
|
||||
inFrontmatter = true
|
||||
continue
|
||||
}
|
||||
|
||||
if inFrontmatter && strings.TrimSpace(line) == "---" {
|
||||
inFrontmatter = false
|
||||
frontmatterEnded = true
|
||||
continue
|
||||
}
|
||||
|
||||
if inFrontmatter {
|
||||
frontmatterLines = append(frontmatterLines, line)
|
||||
} else if frontmatterEnded {
|
||||
bodyLines = append(bodyLines, line)
|
||||
}
|
||||
}
|
||||
|
||||
if err := scanner.Err(); err != nil {
|
||||
return nil, fmt.Errorf("error reading SKILL.md: %w", err)
|
||||
}
|
||||
|
||||
if !frontmatterEnded {
|
||||
return nil, errors.New("SKILL.md frontmatter is not properly closed with ---")
|
||||
}
|
||||
|
||||
// Parse YAML frontmatter
|
||||
frontmatter := strings.Join(frontmatterLines, "\n")
|
||||
repaired, err := UnmarshalSkillFrontmatter(frontmatter, skill)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to parse YAML frontmatter: %w", err)
|
||||
}
|
||||
skill.FrontmatterRepaired = repaired
|
||||
if err := skill.applyInstallName(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Set body instructions
|
||||
skill.Instructions = strings.TrimSpace(strings.Join(bodyLines, "\n"))
|
||||
skill.Loaded = true
|
||||
|
||||
// Validate
|
||||
if err := skill.Validate(); err != nil {
|
||||
return nil, fmt.Errorf("skill validation failed: %w", err)
|
||||
}
|
||||
|
||||
return skill, nil
|
||||
}
|
||||
|
||||
// ParseSkillMetadata parses only the metadata from a SKILL.md file content
|
||||
// This is a lightweight operation for skill discovery (Level 1 only)
|
||||
func ParseSkillMetadata(content string) (*SkillMetadata, error) {
|
||||
skill, err := ParseSkillFile(content)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return skill.ToMetadata(), nil
|
||||
}
|
||||
|
||||
// IsOnDemandInstallerPath reports whether path is a first-use dependency
|
||||
// installer (scripts/install_deps.py and friends). Skills ship these to defer
|
||||
// optional packages to chat time, which cannot work once the skill tree is
|
||||
// snapshotted read-only. The installer agent's prompt ("install these extras
|
||||
// now") and the runtime hint ("skip this installer") key off the same list.
|
||||
func IsOnDemandInstallerPath(scriptPath string) bool {
|
||||
base := strings.ToLower(filepath.Base(strings.TrimSpace(scriptPath)))
|
||||
switch {
|
||||
case strings.Contains(base, "install_dep"):
|
||||
return true
|
||||
case base == "setup_deps.py", base == "bootstrap_deps.py":
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// IsScript checks if a file path represents an executable script
|
||||
func IsScript(path string) bool {
|
||||
ext := strings.ToLower(filepath.Ext(path))
|
||||
scriptExtensions := map[string]bool{
|
||||
".py": true,
|
||||
".sh": true,
|
||||
".bash": true,
|
||||
".js": true,
|
||||
".mjs": true,
|
||||
".cjs": true,
|
||||
".ts": true,
|
||||
".rb": true,
|
||||
".pl": true,
|
||||
".php": true,
|
||||
}
|
||||
return scriptExtensions[ext]
|
||||
}
|
||||
|
||||
// GetScriptLanguage returns the language/interpreter for a script file
|
||||
func GetScriptLanguage(path string) string {
|
||||
ext := strings.ToLower(filepath.Ext(path))
|
||||
languages := map[string]string{
|
||||
".py": "python",
|
||||
".sh": "bash",
|
||||
".bash": "bash",
|
||||
".js": "node",
|
||||
".mjs": "node",
|
||||
".cjs": "node",
|
||||
".ts": "ts-node",
|
||||
".rb": "ruby",
|
||||
".pl": "perl",
|
||||
".php": "php",
|
||||
}
|
||||
if lang, ok := languages[ext]; ok {
|
||||
return lang
|
||||
}
|
||||
return "unknown"
|
||||
}
|
||||
@@ -0,0 +1,203 @@
|
||||
package skills
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"reflect"
|
||||
"regexp"
|
||||
"strings"
|
||||
"unicode"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
// UnmarshalSkillFrontmatter decodes the YAML between SKILL.md's --- markers.
|
||||
//
|
||||
// Third-party skills (ClawHub / SkillHub) often indent `version` / `description`
|
||||
// under `name:` as if it were a nested mapping, or leave a colon unquoted in
|
||||
// a scalar. Strict YAML rejects both with "mapping values are not allowed in
|
||||
// this context". When the first parse fails, those two conservative repairs
|
||||
// are retried so those archives still install; valid frontmatter is unchanged.
|
||||
//
|
||||
// The decode always lands on a temporary value and is copied into dest only
|
||||
// on success, so a failed candidate cannot leave dest half-written.
|
||||
// repaired is true when a repair candidate was what succeeded.
|
||||
func UnmarshalSkillFrontmatter(frontmatter string, dest any) (repaired bool, err error) {
|
||||
firstErr := unmarshalFrontmatterCopy(frontmatter, dest)
|
||||
if firstErr == nil {
|
||||
return false, nil
|
||||
}
|
||||
for _, candidate := range frontmatterRepairCandidates(frontmatter) {
|
||||
if candidate == frontmatter {
|
||||
continue
|
||||
}
|
||||
if unmarshalFrontmatterCopy(candidate, dest) == nil {
|
||||
return true, nil
|
||||
}
|
||||
}
|
||||
return false, firstErr
|
||||
}
|
||||
|
||||
func unmarshalFrontmatterCopy(src string, dest any) error {
|
||||
rv := reflect.ValueOf(dest)
|
||||
if rv.Kind() != reflect.Ptr || rv.IsNil() {
|
||||
return fmt.Errorf("frontmatter dest must be a non-nil pointer")
|
||||
}
|
||||
tmp := reflect.New(rv.Elem().Type())
|
||||
if err := yaml.Unmarshal([]byte(src), tmp.Interface()); err != nil {
|
||||
return err
|
||||
}
|
||||
rv.Elem().Set(tmp.Elem())
|
||||
return nil
|
||||
}
|
||||
|
||||
func frontmatterRepairCandidates(frontmatter string) []string {
|
||||
outdented := repairAccidentalNestedFrontmatter(frontmatter)
|
||||
quoted := quoteColonInUnquotedScalars(frontmatter)
|
||||
both := quoteColonInUnquotedScalars(outdented)
|
||||
return []string{outdented, quoted, both}
|
||||
}
|
||||
|
||||
// repairAccidentalNestedFrontmatter outdents keys that were nested under a
|
||||
// plain scalar, e.g.
|
||||
//
|
||||
// name: 命理大师
|
||||
// version: 1.2.6
|
||||
// description: |
|
||||
//
|
||||
// A real nested mapping (`compatibility:` with no value on the same line) is
|
||||
// left alone because it is valid YAML and does not need this pass.
|
||||
func repairAccidentalNestedFrontmatter(src string) string {
|
||||
lines := strings.Split(src, "\n")
|
||||
out := make([]string, 0, len(lines))
|
||||
for i := 0; i < len(lines); {
|
||||
line := lines[i]
|
||||
out = append(out, line)
|
||||
i++
|
||||
if !isPlainScalarMapping(strings.TrimSpace(line)) {
|
||||
continue
|
||||
}
|
||||
indent := leadingWS(line)
|
||||
if i >= len(lines) {
|
||||
break
|
||||
}
|
||||
next := lines[i]
|
||||
for i < len(lines) && strings.TrimSpace(lines[i]) == "" {
|
||||
out = append(out, lines[i])
|
||||
i++
|
||||
if i < len(lines) {
|
||||
next = lines[i]
|
||||
}
|
||||
}
|
||||
if i >= len(lines) {
|
||||
break
|
||||
}
|
||||
nextIndent := leadingWS(next)
|
||||
if nextIndent <= indent || !looksLikeYAMLKey(strings.TrimSpace(next)) {
|
||||
continue
|
||||
}
|
||||
extra := nextIndent - indent
|
||||
for i < len(lines) {
|
||||
cur := lines[i]
|
||||
if strings.TrimSpace(cur) == "" {
|
||||
out = append(out, cur)
|
||||
i++
|
||||
continue
|
||||
}
|
||||
curIndent := leadingWS(cur)
|
||||
if curIndent < nextIndent {
|
||||
break
|
||||
}
|
||||
out = append(out, stripLeadingWS(cur, extra))
|
||||
i++
|
||||
}
|
||||
}
|
||||
return strings.Join(out, "\n")
|
||||
}
|
||||
|
||||
var unquotedColonScalarRE = regexp.MustCompile(
|
||||
`^(\s*(?:name|description)\s*:\s*)([^"'|>{\[\s#].*:.+)$`,
|
||||
)
|
||||
|
||||
// quoteColonInUnquotedScalars wraps name/description values that contain a
|
||||
// colon so `description: Foo: bar` is not parsed as a nested mapping.
|
||||
func quoteColonInUnquotedScalars(src string) string {
|
||||
lines := strings.Split(src, "\n")
|
||||
for i, line := range lines {
|
||||
m := unquotedColonScalarRE.FindStringSubmatch(line)
|
||||
if len(m) != 3 {
|
||||
continue
|
||||
}
|
||||
value := strings.TrimSpace(m[2])
|
||||
if strings.HasPrefix(value, `"`) || strings.HasPrefix(value, "'") {
|
||||
continue
|
||||
}
|
||||
escaped := strings.ReplaceAll(value, `\`, `\\`)
|
||||
escaped = strings.ReplaceAll(escaped, `"`, `\"`)
|
||||
lines[i] = m[1] + `"` + escaped + `"`
|
||||
}
|
||||
return strings.Join(lines, "\n")
|
||||
}
|
||||
|
||||
func isPlainScalarMapping(trimmed string) bool {
|
||||
if trimmed == "" || strings.HasPrefix(trimmed, "#") {
|
||||
return false
|
||||
}
|
||||
key, value, ok := strings.Cut(trimmed, ":")
|
||||
if !ok || strings.TrimSpace(key) == "" {
|
||||
return false
|
||||
}
|
||||
value = strings.TrimSpace(value)
|
||||
if value == "" {
|
||||
return false
|
||||
}
|
||||
if strings.HasPrefix(value, "#") {
|
||||
return false
|
||||
}
|
||||
if value == "|" || value == ">" ||
|
||||
strings.HasPrefix(value, "|") || strings.HasPrefix(value, ">") ||
|
||||
strings.HasPrefix(value, "{") || strings.HasPrefix(value, "[") {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func looksLikeYAMLKey(trimmed string) bool {
|
||||
if trimmed == "" || strings.HasPrefix(trimmed, "#") || strings.HasPrefix(trimmed, "-") {
|
||||
return false
|
||||
}
|
||||
key, _, ok := strings.Cut(trimmed, ":")
|
||||
if !ok {
|
||||
return false
|
||||
}
|
||||
key = strings.TrimSpace(key)
|
||||
if key == "" {
|
||||
return false
|
||||
}
|
||||
if strings.HasPrefix(key, `"`) || strings.HasPrefix(key, "'") {
|
||||
return true
|
||||
}
|
||||
for i, r := range key {
|
||||
if i == 0 && !unicode.IsLetter(r) && r != '_' {
|
||||
return false
|
||||
}
|
||||
if !unicode.IsLetter(r) && !unicode.IsDigit(r) && r != '_' && r != '-' && r != '.' {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func leadingWS(s string) int {
|
||||
return len(s) - len(strings.TrimLeft(s, " \t"))
|
||||
}
|
||||
|
||||
func stripLeadingWS(s string, n int) string {
|
||||
if n <= 0 {
|
||||
return s
|
||||
}
|
||||
i := 0
|
||||
for i < len(s) && i < n && (s[i] == ' ' || s[i] == '\t') {
|
||||
i++
|
||||
}
|
||||
return s[i:]
|
||||
}
|
||||
@@ -0,0 +1,131 @@
|
||||
package skills
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestParseSkillFileRepairsKeysNestedUnderName(t *testing.T) {
|
||||
content := `---
|
||||
name: 命理大师
|
||||
version: 1.2.6
|
||||
description: |
|
||||
全体系命理大师 — 八字四柱。
|
||||
仅作文化参考。
|
||||
metadata:
|
||||
displayName: "命理大师"
|
||||
version: 1.2.0
|
||||
---
|
||||
# Body
|
||||
`
|
||||
skill, err := ParseSkillFile(content)
|
||||
require.NoError(t, err)
|
||||
require.True(t, skill.FrontmatterRepaired)
|
||||
require.Equal(t, "命理大师", skill.Name)
|
||||
require.Contains(t, skill.Description, "全体系命理大师")
|
||||
require.Contains(t, skill.Instructions, "# Body")
|
||||
}
|
||||
|
||||
func TestParseSkillFileQuotesUnquotedColonInDescription(t *testing.T) {
|
||||
content := `---
|
||||
name: university-applications
|
||||
description: Apply to universities: resume, essays, and interviews
|
||||
---
|
||||
Body
|
||||
`
|
||||
skill, err := ParseSkillFile(content)
|
||||
require.NoError(t, err)
|
||||
require.True(t, skill.FrontmatterRepaired)
|
||||
require.Equal(t, "university-applications", skill.Name)
|
||||
require.Equal(t, "Apply to universities: resume, essays, and interviews", skill.Description)
|
||||
}
|
||||
|
||||
func TestRepairAccidentalNestedFrontmatterLeavesValidNestedMapping(t *testing.T) {
|
||||
src := strings.TrimSpace(`
|
||||
compatibility:
|
||||
python: "3.11"
|
||||
name: pdf-tools
|
||||
description: Extract text
|
||||
`)
|
||||
require.Equal(t, src, repairAccidentalNestedFrontmatter(src))
|
||||
|
||||
var skill Skill
|
||||
repaired, err := UnmarshalSkillFrontmatter(src, &skill)
|
||||
require.NoError(t, err)
|
||||
require.False(t, repaired)
|
||||
require.Equal(t, "pdf-tools", skill.Name)
|
||||
}
|
||||
|
||||
func TestUnmarshalSkillFrontmatterStillRejectsBrokenYAML(t *testing.T) {
|
||||
var skill Skill
|
||||
_, err := UnmarshalSkillFrontmatter(":\n - oops", &skill)
|
||||
require.Error(t, err)
|
||||
require.Empty(t, skill.Name)
|
||||
}
|
||||
|
||||
func TestUnmarshalSkillFrontmatterDoesNotPolluteDestOnFailedRepair(t *testing.T) {
|
||||
type dest struct {
|
||||
Name string `yaml:"name"`
|
||||
Description string `yaml:"description"`
|
||||
Tags []string `yaml:"tags"`
|
||||
}
|
||||
// tags would decode, but the nested name/description plus broken
|
||||
// metadata list is still invalid after the two repairs. dest must
|
||||
// stay zero — not a mix of a partial first attempt and a later one.
|
||||
fm := "tags:\n - a\n - b\nname: alpha\n version: 9.9.9\n description: |\n hello\nmetadata:\n openclaw:\n install:\n - kind: node\n package: iztro\n"
|
||||
var d dest
|
||||
repaired, err := UnmarshalSkillFrontmatter(fm, &d)
|
||||
require.Error(t, err)
|
||||
require.False(t, repaired)
|
||||
require.Empty(t, d.Name)
|
||||
require.Empty(t, d.Tags)
|
||||
}
|
||||
|
||||
func TestParseSkillFileUsesSlugWhenNameIsADisplayTitle(t *testing.T) {
|
||||
content := `---
|
||||
name: Word / DOCX
|
||||
slug: word-docx
|
||||
description: Create and edit Microsoft Word documents.
|
||||
---
|
||||
Body
|
||||
`
|
||||
skill, err := ParseSkillFile(content)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "word-docx", skill.Name)
|
||||
}
|
||||
|
||||
func TestParseSkillFileSlugifiesDisplayTitleWithoutSlug(t *testing.T) {
|
||||
content := `---
|
||||
name: Word / DOCX
|
||||
description: Create and edit Microsoft Word documents.
|
||||
---
|
||||
Body
|
||||
`
|
||||
skill, err := ParseSkillFile(content)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "word-docx", skill.Name)
|
||||
}
|
||||
|
||||
func TestParseSkillFileKeepsUnicodeName(t *testing.T) {
|
||||
content := `---
|
||||
name: 律师助手
|
||||
description: 法律文书助手
|
||||
---
|
||||
Body
|
||||
`
|
||||
skill, err := ParseSkillFile(content)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "律师助手", skill.Name)
|
||||
}
|
||||
|
||||
func TestParseSkillFileRejectsSkillHubFrontmatterWithBrokenExtraYAML(t *testing.T) {
|
||||
content, err := os.ReadFile(filepath.Join("testdata", "skillhub_nested_name.md"))
|
||||
require.NoError(t, err)
|
||||
|
||||
_, err = ParseSkillFile(string(content))
|
||||
require.Error(t, err, "nest repair is not enough when extra keys (metadata.openclaw.install) are still invalid YAML")
|
||||
}
|
||||
@@ -0,0 +1,446 @@
|
||||
package skills
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestParseSkillFile(t *testing.T) {
|
||||
content := `---
|
||||
name: test-skill
|
||||
description: A test skill for unit testing purposes.
|
||||
---
|
||||
# Test Skill
|
||||
|
||||
This is the content of the test skill.
|
||||
|
||||
## Usage
|
||||
|
||||
Use this skill when testing.
|
||||
`
|
||||
|
||||
skill, err := ParseSkillFile(content)
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to parse skill file: %v", err)
|
||||
}
|
||||
|
||||
if skill.Name != "test-skill" {
|
||||
t.Errorf("Expected name 'test-skill', got '%s'", skill.Name)
|
||||
}
|
||||
|
||||
if skill.Description != "A test skill for unit testing purposes." {
|
||||
t.Errorf("Expected description 'A test skill for unit testing purposes.', got '%s'", skill.Description)
|
||||
}
|
||||
|
||||
if skill.Instructions == "" {
|
||||
t.Error("Expected instructions to be non-empty")
|
||||
}
|
||||
|
||||
if !skill.Loaded {
|
||||
t.Error("Expected Loaded to be true after parsing")
|
||||
}
|
||||
if skill.FrontmatterRepaired {
|
||||
t.Error("valid frontmatter should not be marked repaired")
|
||||
}
|
||||
|
||||
t.Logf("Parsed skill: name=%s, description=%s, instructions_len=%d",
|
||||
skill.Name, skill.Description, len(skill.Instructions))
|
||||
}
|
||||
|
||||
func TestSkillValidation(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
skillName string
|
||||
description string
|
||||
wantErr bool
|
||||
errContains string
|
||||
}{
|
||||
{
|
||||
name: "valid skill",
|
||||
skillName: "my-skill",
|
||||
description: "A valid skill",
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
name: "empty name",
|
||||
skillName: "",
|
||||
description: "A skill",
|
||||
wantErr: true,
|
||||
errContains: "name is required",
|
||||
},
|
||||
{
|
||||
name: "invalid characters in name",
|
||||
skillName: "My Skill",
|
||||
description: "A skill",
|
||||
wantErr: true,
|
||||
errContains: "letters, numbers, hyphens, and underscores",
|
||||
},
|
||||
{
|
||||
name: "underscore is allowed",
|
||||
skillName: "word_docx",
|
||||
description: "A skill",
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
name: "unicode letters are allowed",
|
||||
skillName: "律师助手",
|
||||
description: "A skill",
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
name: "reserved word in name",
|
||||
skillName: "my-claude-skill",
|
||||
description: "A skill",
|
||||
wantErr: true,
|
||||
errContains: "reserved word",
|
||||
},
|
||||
{
|
||||
name: "empty description",
|
||||
skillName: "my-skill",
|
||||
description: "",
|
||||
wantErr: true,
|
||||
errContains: "description is required",
|
||||
},
|
||||
{
|
||||
name: "name too long",
|
||||
skillName: "this-is-a-very-long-skill-name-that-exceeds-the-maximum-allowed-length-of-64-characters",
|
||||
description: "A skill",
|
||||
wantErr: true,
|
||||
errContains: "maximum is 64",
|
||||
},
|
||||
{
|
||||
name: "description at 1024 CJK runes is allowed even though it is over 1024 bytes",
|
||||
skillName: "my-skill",
|
||||
description: strings.Repeat("描", 1024),
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
name: "description over 1024 CJK runes is rejected",
|
||||
skillName: "my-skill",
|
||||
description: strings.Repeat("描", 1025),
|
||||
wantErr: true,
|
||||
errContains: "maximum is 1024",
|
||||
},
|
||||
{
|
||||
name: "description over 1024 ASCII runes is rejected",
|
||||
skillName: "my-skill",
|
||||
description: strings.Repeat("a", 1025),
|
||||
wantErr: true,
|
||||
errContains: "maximum is 1024",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
skill := &Skill{
|
||||
Name: tt.skillName,
|
||||
Description: tt.description,
|
||||
}
|
||||
|
||||
err := skill.Validate()
|
||||
if tt.wantErr {
|
||||
if err == nil {
|
||||
t.Errorf("Expected error containing '%s', got nil", tt.errContains)
|
||||
} else if tt.errContains != "" && !containsString(err.Error(), tt.errContains) {
|
||||
t.Errorf("Expected error containing '%s', got '%s'", tt.errContains, err.Error())
|
||||
}
|
||||
} else {
|
||||
if err != nil {
|
||||
t.Errorf("Unexpected error: %v", err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func containsString(s, substr string) bool {
|
||||
return len(s) >= len(substr) && (s == substr || len(s) > 0 && containsSubstring(s, substr))
|
||||
}
|
||||
|
||||
func containsSubstring(s, substr string) bool {
|
||||
for i := 0; i <= len(s)-len(substr); i++ {
|
||||
if s[i:i+len(substr)] == substr {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func TestLoaderDiscoverSkills(t *testing.T) {
|
||||
// Create a temporary skills directory
|
||||
tmpDir, err := os.MkdirTemp("", "skills-test")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to create temp dir: %v", err)
|
||||
}
|
||||
defer os.RemoveAll(tmpDir)
|
||||
|
||||
// Create a test skill directory
|
||||
skillDir := filepath.Join(tmpDir, "test-skill")
|
||||
if err := os.MkdirAll(skillDir, 0755); err != nil {
|
||||
t.Fatalf("Failed to create skill dir: %v", err)
|
||||
}
|
||||
|
||||
// Write SKILL.md
|
||||
skillContent := `---
|
||||
name: test-skill
|
||||
description: A test skill for loader testing.
|
||||
---
|
||||
# Test Skill
|
||||
|
||||
This is the test skill content.
|
||||
`
|
||||
if err := os.WriteFile(filepath.Join(skillDir, "SKILL.md"), []byte(skillContent), 0644); err != nil {
|
||||
t.Fatalf("Failed to write SKILL.md: %v", err)
|
||||
}
|
||||
|
||||
// Create loader and discover skills
|
||||
loader := NewLoader([]string{tmpDir})
|
||||
metadata, err := loader.DiscoverSkills()
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to discover skills: %v", err)
|
||||
}
|
||||
|
||||
if len(metadata) != 1 {
|
||||
t.Fatalf("Expected 1 skill, got %d", len(metadata))
|
||||
}
|
||||
|
||||
if metadata[0].Name != "test-skill" {
|
||||
t.Errorf("Expected skill name 'test-skill', got '%s'", metadata[0].Name)
|
||||
}
|
||||
|
||||
t.Logf("Discovered %d skills: %v", len(metadata), metadata[0].Name)
|
||||
}
|
||||
|
||||
func TestLoaderLoadSkillInstructions(t *testing.T) {
|
||||
// Create a temporary skills directory
|
||||
tmpDir, err := os.MkdirTemp("", "skills-test")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to create temp dir: %v", err)
|
||||
}
|
||||
defer os.RemoveAll(tmpDir)
|
||||
|
||||
// Create a test skill directory
|
||||
skillDir := filepath.Join(tmpDir, "test-skill")
|
||||
if err := os.MkdirAll(skillDir, 0755); err != nil {
|
||||
t.Fatalf("Failed to create skill dir: %v", err)
|
||||
}
|
||||
|
||||
// Write SKILL.md
|
||||
skillContent := `---
|
||||
name: test-skill
|
||||
description: A test skill for content loading.
|
||||
---
|
||||
# Test Skill
|
||||
|
||||
This is the main content.
|
||||
|
||||
## Section 1
|
||||
|
||||
More content here.
|
||||
`
|
||||
if err := os.WriteFile(filepath.Join(skillDir, "SKILL.md"), []byte(skillContent), 0644); err != nil {
|
||||
t.Fatalf("Failed to write SKILL.md: %v", err)
|
||||
}
|
||||
|
||||
// Create loader and load skill instructions
|
||||
loader := NewLoader([]string{tmpDir})
|
||||
skill, err := loader.LoadSkillInstructions("test-skill")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to load skill instructions: %v", err)
|
||||
}
|
||||
|
||||
if skill.Name != "test-skill" {
|
||||
t.Errorf("Expected skill name 'test-skill', got '%s'", skill.Name)
|
||||
}
|
||||
|
||||
if skill.Instructions == "" {
|
||||
t.Error("Expected instructions to be non-empty")
|
||||
}
|
||||
|
||||
if !skill.Loaded {
|
||||
t.Error("Expected Loaded to be true")
|
||||
}
|
||||
|
||||
t.Logf("Loaded skill: name=%s, instructions_len=%d", skill.Name, len(skill.Instructions))
|
||||
}
|
||||
|
||||
func TestLoaderLoadSkillFile(t *testing.T) {
|
||||
// Create a temporary skills directory
|
||||
tmpDir, err := os.MkdirTemp("", "skills-test")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to create temp dir: %v", err)
|
||||
}
|
||||
defer os.RemoveAll(tmpDir)
|
||||
|
||||
// Create a test skill directory with additional files
|
||||
skillDir := filepath.Join(tmpDir, "test-skill")
|
||||
scriptsDir := filepath.Join(skillDir, "scripts")
|
||||
if err := os.MkdirAll(scriptsDir, 0755); err != nil {
|
||||
t.Fatalf("Failed to create skill dir: %v", err)
|
||||
}
|
||||
|
||||
// Write SKILL.md
|
||||
skillContent := `---
|
||||
name: test-skill
|
||||
description: A test skill with additional files.
|
||||
---
|
||||
# Test Skill
|
||||
|
||||
See [GUIDE.md](GUIDE.md) for more info.
|
||||
`
|
||||
if err := os.WriteFile(filepath.Join(skillDir, "SKILL.md"), []byte(skillContent), 0644); err != nil {
|
||||
t.Fatalf("Failed to write SKILL.md: %v", err)
|
||||
}
|
||||
|
||||
// Write additional file
|
||||
guideContent := "# Guide\n\nThis is the guide content."
|
||||
if err := os.WriteFile(filepath.Join(skillDir, "GUIDE.md"), []byte(guideContent), 0644); err != nil {
|
||||
t.Fatalf("Failed to write GUIDE.md: %v", err)
|
||||
}
|
||||
|
||||
// Write a script
|
||||
scriptContent := "#!/usr/bin/env python3\nprint('Hello from script')"
|
||||
if err := os.WriteFile(filepath.Join(scriptsDir, "hello.py"), []byte(scriptContent), 0644); err != nil {
|
||||
t.Fatalf("Failed to write script: %v", err)
|
||||
}
|
||||
|
||||
// Create loader and discover skills first
|
||||
loader := NewLoader([]string{tmpDir})
|
||||
_, err = loader.DiscoverSkills()
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to discover skills: %v", err)
|
||||
}
|
||||
|
||||
// Load additional file
|
||||
file, err := loader.LoadSkillFile("test-skill", "GUIDE.md")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to load skill file: %v", err)
|
||||
}
|
||||
|
||||
if file.Content != guideContent {
|
||||
t.Errorf("Expected guide content, got '%s'", file.Content)
|
||||
}
|
||||
|
||||
if file.IsScript {
|
||||
t.Error("GUIDE.md should not be marked as script")
|
||||
}
|
||||
|
||||
// Load script file
|
||||
scriptFile, err := loader.LoadSkillFile("test-skill", "scripts/hello.py")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to load script file: %v", err)
|
||||
}
|
||||
|
||||
if !scriptFile.IsScript {
|
||||
t.Error("hello.py should be marked as script")
|
||||
}
|
||||
|
||||
t.Logf("Loaded files: GUIDE.md=%d bytes, hello.py=%d bytes (isScript=%v)",
|
||||
len(file.Content), len(scriptFile.Content), scriptFile.IsScript)
|
||||
}
|
||||
|
||||
func TestManagerIntegration(t *testing.T) {
|
||||
// Create a temporary skills directory
|
||||
tmpDir, err := os.MkdirTemp("", "skills-test")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to create temp dir: %v", err)
|
||||
}
|
||||
defer os.RemoveAll(tmpDir)
|
||||
|
||||
// Create a test skill directory
|
||||
skillDir := filepath.Join(tmpDir, "test-skill")
|
||||
if err := os.MkdirAll(skillDir, 0755); err != nil {
|
||||
t.Fatalf("Failed to create skill dir: %v", err)
|
||||
}
|
||||
|
||||
// Write SKILL.md
|
||||
skillContent := `---
|
||||
name: test-skill
|
||||
description: A test skill for manager integration testing.
|
||||
---
|
||||
# Test Skill
|
||||
|
||||
Integration test content.
|
||||
`
|
||||
if err := os.WriteFile(filepath.Join(skillDir, "SKILL.md"), []byte(skillContent), 0644); err != nil {
|
||||
t.Fatalf("Failed to write SKILL.md: %v", err)
|
||||
}
|
||||
|
||||
// Create manager with config
|
||||
config := &ManagerConfig{
|
||||
SkillDirs: []string{tmpDir},
|
||||
AllowedSkills: []string{}, // Allow all
|
||||
Enabled: true,
|
||||
}
|
||||
|
||||
manager := NewManager(config, nil) // No sandbox for this test
|
||||
|
||||
// Initialize
|
||||
ctx := context.Background()
|
||||
if err := manager.Initialize(ctx); err != nil {
|
||||
t.Fatalf("Failed to initialize manager: %v", err)
|
||||
}
|
||||
|
||||
// Get all metadata
|
||||
metadata := manager.GetAllMetadata()
|
||||
if len(metadata) != 1 {
|
||||
t.Fatalf("Expected 1 skill, got %d", len(metadata))
|
||||
}
|
||||
|
||||
// Load skill
|
||||
skill, err := manager.LoadSkill(ctx, "test-skill")
|
||||
if err != nil {
|
||||
t.Fatalf("Failed to load skill: %v", err)
|
||||
}
|
||||
|
||||
if skill.Name != "test-skill" {
|
||||
t.Errorf("Expected skill name 'test-skill', got '%s'", skill.Name)
|
||||
}
|
||||
|
||||
t.Logf("Manager integration test passed: %d skills discovered", len(metadata))
|
||||
}
|
||||
|
||||
func TestIsScript(t *testing.T) {
|
||||
tests := []struct {
|
||||
path string
|
||||
expected bool
|
||||
}{
|
||||
{"script.py", true},
|
||||
{"script.sh", true},
|
||||
{"script.bash", true},
|
||||
{"script.js", true},
|
||||
{"script.mjs", true},
|
||||
{"script.cjs", true},
|
||||
{"script.ts", true},
|
||||
{"script.rb", true},
|
||||
{"README.md", false},
|
||||
{"data.json", false},
|
||||
{"config.yaml", false},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
result := IsScript(tt.path)
|
||||
if result != tt.expected {
|
||||
t.Errorf("IsScript(%s) = %v, expected %v", tt.path, result, tt.expected)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsOnDemandInstallerPath(t *testing.T) {
|
||||
for _, path := range []string{
|
||||
"scripts/install_deps.py",
|
||||
"scripts/install_dep.sh",
|
||||
"setup_deps.py",
|
||||
"bootstrap_deps.py",
|
||||
} {
|
||||
if !IsOnDemandInstallerPath(path) {
|
||||
t.Errorf("IsOnDemandInstallerPath(%s) = false, want true", path)
|
||||
}
|
||||
}
|
||||
if IsOnDemandInstallerPath("scripts/generate_ppt.py") {
|
||||
t.Error("a real skill script must not look like an on-demand installer")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
package skills
|
||||
|
||||
// SkillSource is where one set of skills comes from. Two implementations
|
||||
// exist: the deployment's preloaded skill directories (*Loader) and the
|
||||
// projection of the skills an administrator installed into a workspace sandbox
|
||||
// config's snapshot image (*TenantSkillSource).
|
||||
//
|
||||
// The five methods are the Progressive Disclosure levels the agent asks for:
|
||||
// metadata for the system prompt, the SKILL.md body, individual resource
|
||||
// files, the file listing, and the directory a script runs from.
|
||||
type SkillSource interface {
|
||||
DiscoverSkills() ([]*SkillMetadata, error)
|
||||
LoadSkillInstructions(name string) (*Skill, error)
|
||||
LoadSkillFile(name, relativePath string) (*SkillFile, error)
|
||||
ListSkillFiles(name string) ([]string, error)
|
||||
GetSkillBasePath(name string) (string, error)
|
||||
}
|
||||
|
||||
var _ SkillSource = (*Loader)(nil)
|
||||
|
||||
// imageSkillSource is a source whose skills already exist inside the sandbox
|
||||
// image. It is what separates the two sources at execution time: a preloaded
|
||||
// skill lives on the WeKnora host and has to be uploaded into the sandbox,
|
||||
// while an installed skill is already there and is executed in place.
|
||||
type imageSkillSource interface {
|
||||
SkillSource
|
||||
|
||||
// RemoteScriptPath returns the absolute in-sandbox path of one script.
|
||||
RemoteScriptPath(name, relativePath string) (string, error)
|
||||
}
|
||||
|
||||
var _ imageSkillSource = (*TenantSkillSource)(nil)
|
||||
@@ -0,0 +1,443 @@
|
||||
package skills
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bytes"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
)
|
||||
|
||||
// Bundle limits bound decompression. They mirror the install-time limits: the
|
||||
// archive read here is the same one that was accepted then, and a corrupted or
|
||||
// hostile object in storage must not be able to exhaust this process.
|
||||
const (
|
||||
// maxBundleZipEntries bounds central-directory iteration on the raw
|
||||
// archive, mirroring maxSkillBundleZipEntries. A GitHub zipball carries
|
||||
// the whole repository, so it sits well above the count of files kept as
|
||||
// the skill.
|
||||
maxBundleZipEntries = 100_000
|
||||
// maxBundleEntries caps the files kept as the skill, mirroring
|
||||
// maxSkillBundleFiles. It is counted after directory entries and the
|
||||
// zipball's extra trees are dropped, exactly as the install counts them:
|
||||
// applying it to the raw entry count instead would reject archives the
|
||||
// install accepted, taking read_skill down for a working install.
|
||||
maxBundleEntries = 20_000
|
||||
maxBundleEntryBytes = 32 << 20 // 32 MiB per entry
|
||||
maxBundleBytes = 512 << 20 // 512 MiB across one archive (install cap)
|
||||
)
|
||||
|
||||
// A source lives for a single agent run. read_skill is typically called
|
||||
// several times against the same skill, so a handful of compressed zips
|
||||
// removes the repeated download. 64 MiB is the keep-around budget; a zip
|
||||
// larger than that can stay as the sole occupant so a large skill does not
|
||||
// re-download on every file, without pinning several 100+ MiB archives.
|
||||
const (
|
||||
cachedBundleCount = 4
|
||||
cachedBundleBytes = 64 << 20
|
||||
)
|
||||
|
||||
// TenantSkillSource exposes the skills an administrator installed into one
|
||||
// sandbox config's snapshot image.
|
||||
//
|
||||
// The three disclosure levels come from different places on purpose: name,
|
||||
// description and the SKILL.md body are columns on the row, so the request
|
||||
// path never waits on object storage to tell the model what a skill is, while
|
||||
// individual resource files come from the uploaded archive. Execution uses
|
||||
// neither - the files are already in the image.
|
||||
type TenantSkillSource struct {
|
||||
byName map[string]*types.TenantSkillEntity
|
||||
// order preserves the repository's ordering so the system prompt is stable
|
||||
// between turns.
|
||||
order []string
|
||||
|
||||
// loadBundle fetches the skill zip. It may be nil when the deployment
|
||||
// cannot serve bundles, in which case only the levels backed by the row
|
||||
// are available. Callers that leave install.BundleRef empty should resolve
|
||||
// the catalog object here.
|
||||
loadBundle func(row *types.TenantSkillEntity) ([]byte, error)
|
||||
|
||||
mu sync.Mutex
|
||||
// cache holds downloaded archives, most recently used first, keyed by
|
||||
// bundle_sha256 (or the storage ref). The zip stays compressed: list and
|
||||
// read_skill inflate one entry at a time so a 13k-file skill cannot pin
|
||||
// hundreds of megabytes unpacked for the rest of the turn.
|
||||
cache []cachedBundle
|
||||
}
|
||||
|
||||
type cachedBundle struct {
|
||||
key string
|
||||
archive []byte
|
||||
}
|
||||
|
||||
// NewTenantSkillSource builds a source over the rows of one sandbox config.
|
||||
// Callers pass every row; the source itself decides which are usable.
|
||||
func NewTenantSkillSource(
|
||||
rows []*types.TenantSkillEntity, loadBundle func(row *types.TenantSkillEntity) ([]byte, error),
|
||||
) *TenantSkillSource {
|
||||
src := &TenantSkillSource{
|
||||
byName: make(map[string]*types.TenantSkillEntity, len(rows)),
|
||||
loadBundle: loadBundle,
|
||||
}
|
||||
for _, row := range rows {
|
||||
if !usableSkillRow(row) {
|
||||
continue
|
||||
}
|
||||
if _, exists := src.byName[row.Name]; exists {
|
||||
continue
|
||||
}
|
||||
src.byName[row.Name] = row
|
||||
src.order = append(src.order, row.Name)
|
||||
}
|
||||
return src
|
||||
}
|
||||
|
||||
// usableSkillRow is the one place "the agent can actually run this" is decided
|
||||
// for a single row. A row that is still installing, failed, or was disabled by
|
||||
// an administrator is invisible: telling the model about a skill it cannot
|
||||
// invoke costs it turns and gains nothing.
|
||||
//
|
||||
// The name guard is not defensive: every path this source hands out is
|
||||
// SkillDirFor(row.Name), which joins the name under the skills root. A name
|
||||
// that is not a single path segment yields a path outside the skill, or outside
|
||||
// the root entirely, and those paths reach the model in discovery metadata and
|
||||
// in SkillFile.Path even though execution would refuse them. Filtering the row
|
||||
// out here is what keeps them from ever being spoken. sandbox.IsValidSkillName
|
||||
// is the same rule SkillDirFor enforces, so this cannot disagree with it.
|
||||
func usableSkillRow(row *types.TenantSkillEntity) bool {
|
||||
return row != nil &&
|
||||
row.Enabled &&
|
||||
row.Status == types.SkillStatusReady &&
|
||||
sandbox.IsValidSkillName(row.Name)
|
||||
}
|
||||
|
||||
// DiscoverSkills returns Level 1 metadata for every usable skill.
|
||||
func (s *TenantSkillSource) DiscoverSkills() ([]*SkillMetadata, error) {
|
||||
metadata := make([]*SkillMetadata, 0, len(s.order))
|
||||
for _, name := range s.order {
|
||||
row := s.byName[name]
|
||||
basePath, err := sandbox.SkillDirFor(row.Name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
metadata = append(metadata, &SkillMetadata{
|
||||
Name: row.Name,
|
||||
Description: row.Description,
|
||||
BasePath: basePath,
|
||||
})
|
||||
}
|
||||
return metadata, nil
|
||||
}
|
||||
|
||||
// LoadSkillInstructions returns Level 2 from the row.
|
||||
func (s *TenantSkillSource) LoadSkillInstructions(name string) (*Skill, error) {
|
||||
row, err := s.row(name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
basePath, err := sandbox.SkillDirFor(row.Name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Skill{
|
||||
Name: row.Name,
|
||||
Description: row.Description,
|
||||
BasePath: basePath,
|
||||
FilePath: path.Join(basePath, SkillFileName),
|
||||
Instructions: row.Instructions,
|
||||
Loaded: true,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// LoadSkillFile returns one Level 3 resource out of the uploaded archive.
|
||||
//
|
||||
// The archive is read rather than the image: reading a file out of the image
|
||||
// would need a sandbox, and read_skill must work whether or not this turn has
|
||||
// already booted one.
|
||||
func (s *TenantSkillSource) LoadSkillFile(name, relativePath string) (*SkillFile, error) {
|
||||
row, err := s.row(name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
clean, err := safeSkillRelPath(relativePath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
archive, err := s.bundleArchive(row)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
index, err := skillBundleFileIndex(archive)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
item, ok := index[clean]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("file not found in skill %s: %s", name, relativePath)
|
||||
}
|
||||
content, err := readLimitedSkillZipFile(item)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
basePath, err := sandbox.SkillDirFor(row.Name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &SkillFile{
|
||||
Name: relativePath,
|
||||
// The path the model sees is the in-image one, because that is where
|
||||
// it can act on the file.
|
||||
Path: path.Join(basePath, clean),
|
||||
Content: string(content),
|
||||
IsScript: IsScript(clean),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// ListSkillFiles lists the archive contents of one skill, sorted so repeated
|
||||
// calls within a turn do not look like different answers.
|
||||
func (s *TenantSkillSource) ListSkillFiles(name string) ([]string, error) {
|
||||
row, err := s.row(name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
archive, err := s.bundleArchive(row)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
index, err := skillBundleFileIndex(archive)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
names := make([]string, 0, len(index))
|
||||
for rel := range index {
|
||||
names = append(names, rel)
|
||||
}
|
||||
sort.Strings(names)
|
||||
return names, nil
|
||||
}
|
||||
|
||||
// GetSkillBasePath returns the skill's directory inside the image.
|
||||
func (s *TenantSkillSource) GetSkillBasePath(name string) (string, error) {
|
||||
row, err := s.row(name)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return sandbox.SkillDirFor(row.Name)
|
||||
}
|
||||
|
||||
// RemoteScriptPath returns the absolute in-image path of one script. It is
|
||||
// keyed on the skill name, which is also the directory the installer wrote.
|
||||
//
|
||||
// It deliberately does not consult the archive. The image is what executes,
|
||||
// and a skill whose archive failed to store - the install logs a warning and
|
||||
// carries on - is still installed and runnable.
|
||||
func (s *TenantSkillSource) RemoteScriptPath(name, relativePath string) (string, error) {
|
||||
row, err := s.row(name)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
clean, err := safeSkillRelPath(relativePath)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
basePath, err := sandbox.SkillDirFor(row.Name)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return path.Join(basePath, clean), nil
|
||||
}
|
||||
|
||||
func (s *TenantSkillSource) row(name string) (*types.TenantSkillEntity, error) {
|
||||
if row, ok := s.byName[name]; ok {
|
||||
return row, nil
|
||||
}
|
||||
return nil, fmt.Errorf("skill not found: %s", name)
|
||||
}
|
||||
|
||||
// safeSkillRelPath normalises a caller-supplied relative path and refuses
|
||||
// anything that leaves the skill directory.
|
||||
func safeSkillRelPath(relativePath string) (string, error) {
|
||||
trimmed := strings.TrimSpace(relativePath)
|
||||
if trimmed == "" {
|
||||
return "", fmt.Errorf("skill file path is required")
|
||||
}
|
||||
if path.IsAbs(trimmed) {
|
||||
return "", fmt.Errorf("invalid skill file path: %s", relativePath)
|
||||
}
|
||||
clean := path.Clean(trimmed)
|
||||
if clean == "." || clean == ".." || strings.HasPrefix(clean, "../") {
|
||||
return "", fmt.Errorf("invalid skill file path: %s", relativePath)
|
||||
}
|
||||
return clean, nil
|
||||
}
|
||||
|
||||
// bundleArchive returns the compressed zip of one skill, downloading it at
|
||||
// most once per cache lifetime.
|
||||
func (s *TenantSkillSource) bundleArchive(
|
||||
row *types.TenantSkillEntity,
|
||||
) ([]byte, error) {
|
||||
if s.loadBundle == nil {
|
||||
return nil, fmt.Errorf("skill bundles are not available in this deployment")
|
||||
}
|
||||
|
||||
key := strings.TrimSpace(row.BundleSHA256)
|
||||
if key == "" {
|
||||
key = strings.TrimSpace(row.BundleRef)
|
||||
}
|
||||
if key == "" {
|
||||
key = strings.TrimSpace(row.CatalogID)
|
||||
}
|
||||
if key == "" {
|
||||
key = strings.TrimSpace(row.ID)
|
||||
}
|
||||
if archive := s.cached(key); archive != nil {
|
||||
return archive, nil
|
||||
}
|
||||
|
||||
archive, err := s.loadBundle(row)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("download bundle of skill %s: %w", row.Name, err)
|
||||
}
|
||||
if len(archive) == 0 {
|
||||
return nil, fmt.Errorf(
|
||||
"skill %s has no stored bundle; its files cannot be read", row.Name)
|
||||
}
|
||||
s.store(key, archive)
|
||||
return archive, nil
|
||||
}
|
||||
|
||||
func (s *TenantSkillSource) cached(key string) []byte {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
for i, entry := range s.cache {
|
||||
if entry.key != key {
|
||||
continue
|
||||
}
|
||||
s.cache = append(s.cache[:i], s.cache[i+1:]...)
|
||||
s.cache = append([]cachedBundle{entry}, s.cache...)
|
||||
return entry.archive
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *TenantSkillSource) store(key string, archive []byte) {
|
||||
if key == "" || len(archive) == 0 || len(archive) > maxBundleBytes {
|
||||
return
|
||||
}
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
for i, entry := range s.cache {
|
||||
if entry.key != key {
|
||||
continue
|
||||
}
|
||||
s.cache = append(s.cache[:i], s.cache[i+1:]...)
|
||||
break
|
||||
}
|
||||
s.cache = append([]cachedBundle{{key: key, archive: archive}}, s.cache...)
|
||||
for len(s.cache) > 1 &&
|
||||
(len(s.cache) > cachedBundleCount || s.cachedBytes() > cachedBundleBytes) {
|
||||
s.cache = s.cache[:len(s.cache)-1]
|
||||
}
|
||||
}
|
||||
|
||||
func (s *TenantSkillSource) cachedBytes() int {
|
||||
total := 0
|
||||
for _, entry := range s.cache {
|
||||
total += len(entry.archive)
|
||||
}
|
||||
return total
|
||||
}
|
||||
|
||||
// skillBundleFileIndex maps skill-root-relative paths to zip entries without
|
||||
// inflating file bodies.
|
||||
func skillBundleFileIndex(archive []byte) (map[string]*zip.File, error) {
|
||||
reader, err := zip.NewReader(bytes.NewReader(archive), int64(len(archive)))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("not a readable zip archive: %w", err)
|
||||
}
|
||||
if len(reader.File) > maxBundleZipEntries {
|
||||
return nil, fmt.Errorf("archive holds more than %d entries", maxBundleZipEntries)
|
||||
}
|
||||
|
||||
raw := make(map[string]*zip.File, len(reader.File))
|
||||
for _, entry := range reader.File {
|
||||
info := entry.FileInfo()
|
||||
if info.IsDir() || info.Mode()&os.ModeSymlink != 0 {
|
||||
continue
|
||||
}
|
||||
name := path.Clean(entry.Name)
|
||||
if name == "." || path.IsAbs(name) ||
|
||||
name == ".." || strings.HasPrefix(name, "../") {
|
||||
return nil, fmt.Errorf("entry %q escapes the archive root", entry.Name)
|
||||
}
|
||||
raw[name] = entry
|
||||
}
|
||||
prefix, err := skillZipRootPrefix(raw)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := raw
|
||||
if prefix != "" {
|
||||
out = make(map[string]*zip.File, len(raw))
|
||||
for name, entry := range raw {
|
||||
if !strings.HasPrefix(name, prefix+"/") {
|
||||
continue
|
||||
}
|
||||
out[strings.TrimPrefix(name, prefix+"/")] = entry
|
||||
}
|
||||
if _, ok := out[SkillFileName]; !ok {
|
||||
return nil, fmt.Errorf("%s is missing from the archive", SkillFileName)
|
||||
}
|
||||
}
|
||||
if len(out) > maxBundleEntries {
|
||||
return nil, fmt.Errorf("archive holds more than %d files", maxBundleEntries)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func skillZipRootPrefix(raw map[string]*zip.File) (string, error) {
|
||||
if _, ok := raw[SkillFileName]; ok {
|
||||
return "", nil
|
||||
}
|
||||
prefix := ""
|
||||
for name := range raw {
|
||||
if path.Base(name) != SkillFileName {
|
||||
continue
|
||||
}
|
||||
dir := path.Dir(name)
|
||||
if dir == "." || strings.Contains(dir, "/") {
|
||||
continue
|
||||
}
|
||||
if prefix != "" && prefix != dir {
|
||||
return "", fmt.Errorf("archive holds more than one skill")
|
||||
}
|
||||
prefix = dir
|
||||
}
|
||||
if prefix == "" {
|
||||
return "", fmt.Errorf("%s is missing from the archive", SkillFileName)
|
||||
}
|
||||
return prefix, nil
|
||||
}
|
||||
|
||||
func readLimitedSkillZipFile(entry *zip.File) ([]byte, error) {
|
||||
rc, err := entry.Open()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("cannot read %q: %w", entry.Name, err)
|
||||
}
|
||||
content, err := io.ReadAll(io.LimitReader(rc, maxBundleEntryBytes+1))
|
||||
_ = rc.Close()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("cannot read %q: %w", entry.Name, err)
|
||||
}
|
||||
if len(content) > maxBundleEntryBytes {
|
||||
return nil, fmt.Errorf("entry %q is too large", entry.Name)
|
||||
}
|
||||
return content, nil
|
||||
}
|
||||
@@ -0,0 +1,453 @@
|
||||
package skills
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/Tencent/WeKnora/internal/sandbox"
|
||||
"github.com/Tencent/WeKnora/internal/types"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestTenantSkillSourceOnlyExposesUsableSkills(t *testing.T) {
|
||||
rows := []*types.TenantSkillEntity{
|
||||
{ID: "sk-1", Name: "ready-enabled", Status: types.SkillStatusReady, Enabled: true},
|
||||
{ID: "sk-2", Name: "ready-disabled", Status: types.SkillStatusReady, Enabled: false},
|
||||
{ID: "sk-3", Name: "still-installing", Status: types.SkillStatusInstalling, Enabled: true},
|
||||
{ID: "sk-4", Name: "failed", Status: types.SkillStatusFailed, Enabled: true},
|
||||
}
|
||||
src := NewTenantSkillSource(rows, nil)
|
||||
|
||||
metadata, err := src.DiscoverSkills()
|
||||
|
||||
require.NoError(t, err)
|
||||
require.Len(t, metadata, 1,
|
||||
"a skill the agent cannot actually run must never reach the system prompt")
|
||||
require.Equal(t, "ready-enabled", metadata[0].Name)
|
||||
}
|
||||
|
||||
// The name is joined under the skills root to build every path this source
|
||||
// hands out, so it has to be a single path segment. Execution rejects an
|
||||
// escaping path anyway, but the base path travels to the model in metadata
|
||||
// and in SkillFile.Path, which is the same reasoning guardSkillDir applies
|
||||
// on the install side.
|
||||
func TestTenantSkillSourceRejectsANameThatIsNotOnePathSegment(t *testing.T) {
|
||||
src := NewTenantSkillSource([]*types.TenantSkillEntity{
|
||||
{ID: "sk-2", Name: "../escaping", Status: types.SkillStatusReady, Enabled: true},
|
||||
{ID: "sk-3", Name: "nested/name", Status: types.SkillStatusReady, Enabled: true},
|
||||
{ID: "sk-4", Name: "..", Status: types.SkillStatusReady, Enabled: true},
|
||||
}, nil)
|
||||
|
||||
metadata, err := src.DiscoverSkills()
|
||||
require.NoError(t, err)
|
||||
require.Empty(t, metadata)
|
||||
for _, name := range []string{"../escaping", "nested/name", ".."} {
|
||||
_, err := src.GetSkillBasePath(name)
|
||||
require.Error(t, err, "%s must be invisible, not merely unexecutable", name)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTenantSkillSourceBasePathIsTheImageDir(t *testing.T) {
|
||||
src := NewTenantSkillSource([]*types.TenantSkillEntity{
|
||||
{ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true},
|
||||
}, nil)
|
||||
|
||||
base, err := src.GetSkillBasePath("pdf")
|
||||
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "/opt/weknora/tenant/skills/pdf", base,
|
||||
"the path is the skill name: that is the directory the installer writes")
|
||||
}
|
||||
|
||||
// A skill the source does not expose must be unreachable through every entry
|
||||
// point, not just discovery: routing decisions elsewhere ask these methods.
|
||||
func TestTenantSkillSourceHidesUnusableSkillsFromEveryLookup(t *testing.T) {
|
||||
src := NewTenantSkillSource([]*types.TenantSkillEntity{
|
||||
{ID: "sk-2", Name: "ready-disabled", Status: types.SkillStatusReady, Enabled: false},
|
||||
}, func(*types.TenantSkillEntity) ([]byte, error) { return nil, errors.New("must not be called") })
|
||||
|
||||
_, err := src.GetSkillBasePath("ready-disabled")
|
||||
require.Error(t, err)
|
||||
_, err = src.LoadSkillInstructions("ready-disabled")
|
||||
require.Error(t, err)
|
||||
_, err = src.ListSkillFiles("ready-disabled")
|
||||
require.Error(t, err)
|
||||
_, err = src.LoadSkillFile("ready-disabled", "scripts/run.py")
|
||||
require.Error(t, err)
|
||||
_, err = src.RemoteScriptPath("ready-disabled", "scripts/run.py")
|
||||
require.Error(t, err)
|
||||
}
|
||||
|
||||
// Level 2 comes from the database projection: the request path must not depend
|
||||
// on downloading an archive to answer "what does this skill say".
|
||||
func TestTenantSkillSourceLoadsInstructionsFromTheRow(t *testing.T) {
|
||||
src := NewTenantSkillSource([]*types.TenantSkillEntity{{
|
||||
ID: "sk-1", Name: "pdf", Description: "PDF helpers",
|
||||
Instructions: "Run scripts/extract.py.",
|
||||
Status: types.SkillStatusReady, Enabled: true,
|
||||
}}, func(*types.TenantSkillEntity) ([]byte, error) { return nil, errors.New("must not be called") })
|
||||
|
||||
skill, err := src.LoadSkillInstructions("pdf")
|
||||
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "pdf", skill.Name)
|
||||
require.Equal(t, "PDF helpers", skill.Description)
|
||||
require.Equal(t, "Run scripts/extract.py.", skill.Instructions)
|
||||
require.True(t, skill.Loaded)
|
||||
require.Equal(t, "/opt/weknora/tenant/skills/pdf", skill.BasePath)
|
||||
require.Equal(t, "/opt/weknora/tenant/skills/pdf/SKILL.md", skill.FilePath)
|
||||
}
|
||||
|
||||
func TestTenantSkillSourceReadsLevel3FilesFromTheBundle(t *testing.T) {
|
||||
archive := zipArchive(t, map[string]string{
|
||||
// Wrapped in a top-level directory, which is how people upload.
|
||||
"pdf-tools/SKILL.md": "---\nname: pdf\ndescription: d\n---\nbody\n",
|
||||
"pdf-tools/reference/FORMS.md": "form notes",
|
||||
"pdf-tools/scripts/extract.py": "print('hi')\n",
|
||||
})
|
||||
src := NewTenantSkillSource([]*types.TenantSkillEntity{{
|
||||
ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true,
|
||||
BundleRef: "local://sk-1.zip", BundleSHA256: "sha-1",
|
||||
}}, func(row *types.TenantSkillEntity) ([]byte, error) {
|
||||
require.Equal(t, "local://sk-1.zip", row.BundleRef)
|
||||
return archive, nil
|
||||
})
|
||||
|
||||
files, err := src.ListSkillFiles("pdf")
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, []string{"SKILL.md", "reference/FORMS.md", "scripts/extract.py"}, files)
|
||||
|
||||
file, err := src.LoadSkillFile("pdf", "reference/FORMS.md")
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "form notes", file.Content)
|
||||
require.Equal(t, "/opt/weknora/tenant/skills/pdf/reference/FORMS.md", file.Path,
|
||||
"the path the model is shown must be the one it can execute or read in the sandbox")
|
||||
require.False(t, file.IsScript)
|
||||
|
||||
script, err := src.LoadSkillFile("pdf", "scripts/extract.py")
|
||||
require.NoError(t, err)
|
||||
require.True(t, script.IsScript)
|
||||
}
|
||||
|
||||
func TestTenantSkillSourceDownloadsEachBundleOnce(t *testing.T) {
|
||||
archive := zipArchive(t, map[string]string{
|
||||
"SKILL.md": "---\nname: pdf\ndescription: d\n---\nbody\n",
|
||||
"scripts/extract.py": "print('hi')\n",
|
||||
})
|
||||
downloads := 0
|
||||
src := NewTenantSkillSource([]*types.TenantSkillEntity{{
|
||||
ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true,
|
||||
BundleRef: "local://sk-1.zip", BundleSHA256: "sha-1",
|
||||
}}, func(*types.TenantSkillEntity) ([]byte, error) {
|
||||
downloads++
|
||||
return archive, nil
|
||||
})
|
||||
|
||||
_, err := src.ListSkillFiles("pdf")
|
||||
require.NoError(t, err)
|
||||
_, err = src.LoadSkillFile("pdf", "scripts/extract.py")
|
||||
require.NoError(t, err)
|
||||
|
||||
require.Equal(t, 1, downloads,
|
||||
"one read_skill call per file would re-download the whole archive every time")
|
||||
}
|
||||
|
||||
func TestTenantSkillSourceRefusesPathsOutsideTheSkill(t *testing.T) {
|
||||
src := NewTenantSkillSource([]*types.TenantSkillEntity{{
|
||||
ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true,
|
||||
BundleRef: "local://sk-1.zip",
|
||||
}}, func(*types.TenantSkillEntity) ([]byte, error) {
|
||||
return nil, errors.New("must not be called")
|
||||
})
|
||||
|
||||
for _, rel := range []string{"../other/secret", "/etc/passwd", ""} {
|
||||
_, err := src.LoadSkillFile("pdf", rel)
|
||||
require.Error(t, err, "rel=%q", rel)
|
||||
_, err = src.RemoteScriptPath("pdf", rel)
|
||||
require.Error(t, err, "rel=%q", rel)
|
||||
}
|
||||
}
|
||||
|
||||
// A skill whose archive could not be stored is still installed and runnable in
|
||||
// the image, so execution must not depend on the bundle being downloadable.
|
||||
func TestTenantSkillSourceReportsAMissingBundleWithoutBlockingExecution(t *testing.T) {
|
||||
src := NewTenantSkillSource([]*types.TenantSkillEntity{{
|
||||
ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true,
|
||||
}}, nil)
|
||||
|
||||
_, err := src.LoadSkillFile("pdf", "scripts/extract.py")
|
||||
require.Error(t, err)
|
||||
|
||||
remote, err := src.RemoteScriptPath("pdf", "scripts/extract.py")
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "/opt/weknora/tenant/skills/pdf/scripts/extract.py", remote)
|
||||
}
|
||||
|
||||
func TestManagerIgnoresPreloadedSkillsWhenTenantSourceIsAttached(t *testing.T) {
|
||||
dir := preloadedSkillDir(t, "document-analyzer", "preloaded description")
|
||||
mgr := NewManager(&ManagerConfig{SkillDirs: []string{dir}, Enabled: true}, nil)
|
||||
mgr.WithTenantSource(NewTenantSkillSource([]*types.TenantSkillEntity{
|
||||
{
|
||||
ID: "sk-1", Name: "pdf", Description: "tenant description",
|
||||
Instructions: "tenant body", Status: types.SkillStatusReady, Enabled: true,
|
||||
},
|
||||
{
|
||||
ID: "sk-2", Name: "csv", Description: "tenant only",
|
||||
Status: types.SkillStatusReady, Enabled: true,
|
||||
},
|
||||
}, nil))
|
||||
|
||||
require.NoError(t, mgr.Initialize(context.Background()))
|
||||
|
||||
byName := map[string]*SkillMetadata{}
|
||||
for _, meta := range mgr.GetAllMetadata() {
|
||||
byName[meta.Name] = meta
|
||||
}
|
||||
require.Len(t, byName, 2)
|
||||
require.NotContains(t, byName, "document-analyzer",
|
||||
"host preloaded skills are not in the sandbox image")
|
||||
require.Equal(t, "tenant description", byName["pdf"].Description)
|
||||
require.Equal(t, "tenant only", byName["csv"].Description)
|
||||
|
||||
skill, err := mgr.LoadSkill(context.Background(), "pdf")
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "tenant body", skill.Instructions)
|
||||
|
||||
_, err = mgr.LoadSkill(context.Background(), "document-analyzer")
|
||||
require.Error(t, err, "a host-only skill must not be readable once the image is the source")
|
||||
}
|
||||
|
||||
func TestManagerRunsATenantSkillFromTheImageWithoutUploading(t *testing.T) {
|
||||
sandboxMgr := &recordingSandboxManager{}
|
||||
mgr := NewManager(&ManagerConfig{SkillDirs: nil, Enabled: true}, sandboxMgr)
|
||||
mgr.WithTenantSource(NewTenantSkillSource([]*types.TenantSkillEntity{{
|
||||
ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true,
|
||||
}}, nil))
|
||||
require.NoError(t, mgr.Initialize(context.Background()))
|
||||
|
||||
_, err := mgr.ExecuteScript(
|
||||
types.WithSessionID(context.Background(), "sess-1"),
|
||||
"pdf", "scripts/extract.py", []string{"--flag"}, "",
|
||||
)
|
||||
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, sandboxMgr.config)
|
||||
require.Equal(t, "/opt/weknora/tenant/skills/pdf/scripts/extract.py",
|
||||
sandboxMgr.config.RemoteScriptPath)
|
||||
require.Empty(t, sandboxMgr.config.Script,
|
||||
"there is no host-side copy of an installed skill to upload")
|
||||
require.Equal(t, []string{"--flag"}, sandboxMgr.config.Args)
|
||||
require.Equal(t, "sess-1", sandboxMgr.config.SessionID)
|
||||
require.Equal(t, sandbox.SkillsImageRoot+"/pdf",
|
||||
sandboxMgr.config.Env["WEKNORA_SKILL_DIR"])
|
||||
require.Equal(t, "/workspace/output", sandboxMgr.config.Env[artifactOutputEnvVar])
|
||||
require.Equal(t, sandbox.SessionSkillPackageDir("pdf"),
|
||||
sandboxMgr.config.Env["PYTHONPATH"])
|
||||
require.Equal(t, sandbox.SessionSkillPackageDir("pdf"),
|
||||
sandboxMgr.config.Env["NODE_PATH"])
|
||||
}
|
||||
|
||||
func TestManagerRunsAWorkspaceScriptWithInstalledSkillInterpreter(t *testing.T) {
|
||||
sandboxMgr := &recordingSandboxManager{}
|
||||
mgr := NewManager(&ManagerConfig{SkillDirs: nil, Enabled: true}, sandboxMgr)
|
||||
mgr.WithTenantSource(NewTenantSkillSource([]*types.TenantSkillEntity{{
|
||||
ID: "sk-1", Name: "ppt-generator", Status: types.SkillStatusReady, Enabled: true,
|
||||
}}, nil))
|
||||
require.NoError(t, mgr.Initialize(context.Background()))
|
||||
|
||||
_, err := mgr.ExecuteScript(
|
||||
types.WithSessionID(context.Background(), "sess-1"),
|
||||
"ppt-generator", "/workspace/output/generate_rencui_ppt.py", []string{"--theme", "doraemon"}, "",
|
||||
)
|
||||
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, sandboxMgr.config)
|
||||
require.Equal(t, "/workspace/output/generate_rencui_ppt.py", sandboxMgr.config.RemoteScriptPath)
|
||||
require.Equal(t, sandbox.SkillsImageRoot+"/ppt-generator", sandboxMgr.config.SkillDir)
|
||||
require.Empty(t, sandboxMgr.config.Script)
|
||||
require.Equal(t, []string{"--theme", "doraemon"}, sandboxMgr.config.Args)
|
||||
require.Equal(t, sandbox.SkillsImageRoot+"/ppt-generator",
|
||||
sandboxMgr.config.Env["WEKNORA_SKILL_DIR"])
|
||||
}
|
||||
|
||||
func TestManagerRejectsWorkspaceScriptForPreloadedSkill(t *testing.T) {
|
||||
dir := preloadedSkillDir(t, "pdf", "preloaded description")
|
||||
sandboxMgr := &recordingSandboxManager{}
|
||||
mgr := NewManager(&ManagerConfig{SkillDirs: []string{dir}, Enabled: true}, sandboxMgr)
|
||||
require.NoError(t, mgr.Initialize(context.Background()))
|
||||
|
||||
_, err := mgr.ExecuteScript(context.Background(), "pdf", "/workspace/output/custom.py", nil, "")
|
||||
require.Error(t, err)
|
||||
require.Contains(t, err.Error(), "shell_exec")
|
||||
require.Zero(t, sandboxMgr.calls)
|
||||
}
|
||||
|
||||
func TestManagerRejectsWorkspaceInputAsSkillScript(t *testing.T) {
|
||||
sandboxMgr := &recordingSandboxManager{}
|
||||
mgr := NewManager(&ManagerConfig{SkillDirs: nil, Enabled: true}, sandboxMgr)
|
||||
mgr.WithTenantSource(NewTenantSkillSource([]*types.TenantSkillEntity{{
|
||||
ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true,
|
||||
}}, nil))
|
||||
require.NoError(t, mgr.Initialize(context.Background()))
|
||||
|
||||
_, err := mgr.ExecuteScript(context.Background(), "pdf", "/workspace/input/upload.py", nil, "")
|
||||
require.Error(t, err)
|
||||
require.Zero(t, sandboxMgr.calls)
|
||||
}
|
||||
|
||||
// The in-sandbox directory is what read_skill shows the model so it can name
|
||||
// the skill's own interpreter instead of probing the system python3. A
|
||||
// preloaded skill has no such directory: its base path is on the WeKnora host,
|
||||
// which no command in the sandbox can reach.
|
||||
func TestSandboxSkillDirOnlyAnswersForInstalledSkills(t *testing.T) {
|
||||
installed := NewManager(&ManagerConfig{Enabled: true}, nil)
|
||||
installed.WithTenantSource(NewTenantSkillSource([]*types.TenantSkillEntity{{
|
||||
ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true,
|
||||
}}, nil))
|
||||
require.NoError(t, installed.Initialize(context.Background()))
|
||||
|
||||
dir, ok := installed.SandboxSkillDir("pdf")
|
||||
require.True(t, ok)
|
||||
require.Equal(t, sandbox.SkillsImageRoot+"/pdf", dir)
|
||||
|
||||
preloaded := NewManager(&ManagerConfig{
|
||||
SkillDirs: []string{preloadedSkillDir(t, "pdf", "preloaded description")},
|
||||
Enabled: true,
|
||||
}, nil)
|
||||
require.NoError(t, preloaded.Initialize(context.Background()))
|
||||
|
||||
_, ok = preloaded.SandboxSkillDir("pdf")
|
||||
require.False(t, ok)
|
||||
}
|
||||
|
||||
// Preloaded skills keep uploading from the host and keep running in their own
|
||||
// directory; the tenant source must not change that path at all.
|
||||
func TestManagerKeepsPreloadedSkillExecutionWhenNoTenantSource(t *testing.T) {
|
||||
dir := preloadedSkillDir(t, "pdf", "preloaded description")
|
||||
sandboxMgr := &recordingSandboxManager{}
|
||||
mgr := NewManager(&ManagerConfig{SkillDirs: []string{dir}, Enabled: true}, sandboxMgr)
|
||||
require.NoError(t, mgr.Initialize(context.Background()))
|
||||
|
||||
_, err := mgr.ExecuteScript(context.Background(), "pdf", "scripts/run.py", nil, "")
|
||||
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, sandboxMgr.config)
|
||||
require.Empty(t, sandboxMgr.config.RemoteScriptPath)
|
||||
require.Equal(t, filepath.Join(dir, "pdf", "scripts", "run.py"), sandboxMgr.config.Script)
|
||||
require.Equal(t, dir+"/pdf", sandboxMgr.config.WorkDir)
|
||||
}
|
||||
|
||||
type recordingSandboxManager struct {
|
||||
config *sandbox.ExecuteConfig
|
||||
calls int
|
||||
}
|
||||
|
||||
func (m *recordingSandboxManager) Execute(
|
||||
_ context.Context, config *sandbox.ExecuteConfig,
|
||||
) (*sandbox.ExecuteResult, error) {
|
||||
m.calls++
|
||||
m.config = config
|
||||
return &sandbox.ExecuteResult{ExitCode: 0}, nil
|
||||
}
|
||||
|
||||
func (m *recordingSandboxManager) Cleanup(context.Context) error { return nil }
|
||||
func (m *recordingSandboxManager) GetSandbox() sandbox.Sandbox { return nil }
|
||||
func (m *recordingSandboxManager) GetType() sandbox.SandboxType { return sandbox.SandboxTypeCube }
|
||||
|
||||
// preloadedSkillDir writes one deployment-preloaded skill to a temp directory
|
||||
// and returns the search root it lives under.
|
||||
func preloadedSkillDir(t *testing.T, name, description string) string {
|
||||
t.Helper()
|
||||
root := t.TempDir()
|
||||
skillDir := filepath.Join(root, name)
|
||||
require.NoError(t, os.MkdirAll(filepath.Join(skillDir, "scripts"), 0o755))
|
||||
require.NoError(t, os.WriteFile(filepath.Join(skillDir, SkillFileName),
|
||||
[]byte("---\nname: "+name+"\ndescription: "+description+"\n---\npreloaded body\n"), 0o644))
|
||||
require.NoError(t, os.WriteFile(filepath.Join(skillDir, "scripts", "run.py"),
|
||||
[]byte("print('preloaded')\n"), 0o644))
|
||||
return root
|
||||
}
|
||||
|
||||
func TestTenantSkillSourceCacheKeepsOneOversizeArchive(t *testing.T) {
|
||||
src := NewTenantSkillSource(nil, nil)
|
||||
src.store("small", bytes.Repeat([]byte("s"), 32))
|
||||
src.store("big", bytes.Repeat([]byte("b"), cachedBundleBytes+1))
|
||||
|
||||
require.Equal(t, bytes.Repeat([]byte("b"), cachedBundleBytes+1), src.cached("big"))
|
||||
require.Nil(t, src.cached("small"),
|
||||
"a zip over the keep-around budget must not sit next to other entries")
|
||||
}
|
||||
|
||||
// The agent reads the same archive the install accepted, so the two have to
|
||||
// count it the same way. Counting raw zip entries against the skill-file cap
|
||||
// rejects bundles the install took — directory entries alone can carry a real
|
||||
// skill past 20k — and read_skill then fails on an install that works.
|
||||
const bundleIndexSkillMD = "---\nname: pdf\ndescription: d\n---\nbody\n"
|
||||
|
||||
func TestSkillBundleFileIndexCountsFilesTheWayTheInstallDid(t *testing.T) {
|
||||
files := map[string]string{"repo-main/" + SkillFileName: bundleIndexSkillMD}
|
||||
for i := 0; i < maxBundleEntries-1; i++ {
|
||||
files[fmt.Sprintf("repo-main/templates/asset-%d.txt", i)] = ""
|
||||
}
|
||||
// Directory entries push the raw count past the skill-file cap without
|
||||
// adding a single file the skill is made of.
|
||||
dirs := make([]string, 0, 2000)
|
||||
for i := 0; i < 2000; i++ {
|
||||
dirs = append(dirs, fmt.Sprintf("repo-main/templates/dir-%d/", i))
|
||||
}
|
||||
|
||||
index, err := skillBundleFileIndex(zipArchiveWithDirs(t, files, dirs))
|
||||
|
||||
require.NoError(t, err)
|
||||
require.Len(t, index, maxBundleEntries)
|
||||
require.Contains(t, index, SkillFileName)
|
||||
}
|
||||
|
||||
func TestSkillBundleFileIndexRejectsMoreSkillFilesThanTheCap(t *testing.T) {
|
||||
files := map[string]string{"repo-main/" + SkillFileName: bundleIndexSkillMD}
|
||||
for i := 0; i < maxBundleEntries; i++ {
|
||||
files[fmt.Sprintf("repo-main/templates/asset-%d.txt", i)] = ""
|
||||
}
|
||||
|
||||
_, err := skillBundleFileIndex(zipArchiveWithDirs(t, files, nil))
|
||||
|
||||
require.ErrorContains(t, err, "more than 20000 files")
|
||||
}
|
||||
|
||||
func zipArchiveWithDirs(t *testing.T, files map[string]string, dirs []string) []byte {
|
||||
t.Helper()
|
||||
var buf bytes.Buffer
|
||||
writer := zip.NewWriter(&buf)
|
||||
for _, name := range dirs {
|
||||
_, err := writer.Create(name)
|
||||
require.NoError(t, err)
|
||||
}
|
||||
for name, content := range files {
|
||||
entry, err := writer.Create(name)
|
||||
require.NoError(t, err)
|
||||
_, err = entry.Write([]byte(content))
|
||||
require.NoError(t, err)
|
||||
}
|
||||
require.NoError(t, writer.Close())
|
||||
return buf.Bytes()
|
||||
}
|
||||
|
||||
func zipArchive(t *testing.T, files map[string]string) []byte {
|
||||
t.Helper()
|
||||
var buf bytes.Buffer
|
||||
writer := zip.NewWriter(&buf)
|
||||
for name, content := range files {
|
||||
entry, err := writer.Create(name)
|
||||
require.NoError(t, err)
|
||||
_, err = entry.Write([]byte(content))
|
||||
require.NoError(t, err)
|
||||
}
|
||||
require.NoError(t, writer.Close())
|
||||
return buf.Bytes()
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
---
|
||||
name: 命理大师
|
||||
version: 1.2.6
|
||||
description: |
|
||||
全体系命理大师 — 八字四柱、紫微斗数、奇门遁甲、六爻、梅花易数、塔罗、星盘、
|
||||
数字命理、九宫飞星风水、掌纹面相、起名命名、穿衣搭配、合婚择吉一站式解读。仅作文化参考,不替代医疗、法律、心理、财务、婚姻、命名等
|
||||
专业建议;遇重大决策请咨询专业人士。
|
||||
【数据与隐私提示】本 skill 会在本地文件系统保存用户主动提交的出生年月日时、地点、
|
||||
姓名、可选的家庭成员信息、交互主题记录(用于追踪偏好,默认关闭),以及推送任务日志。
|
||||
所有数据均不上传,仅在本地 data/ 目录留存;用户可随时通过 profile.js 查看、编辑或删除。
|
||||
每日推送为 opt-in,默认关闭。
|
||||
metadata:
|
||||
displayName: "命理大师"
|
||||
author:
|
||||
- "腾讯高级研发-enoyao"
|
||||
- "腾讯高级产品运营-rekyhe"
|
||||
version: 1.2.0
|
||||
keywords: 八字, 紫微斗数, 奇门遁甲, 六爻, 梅花易数, 塔罗, 星盘, 九宫飞星, 掌纹, 手相, 面相, 起名, 命名, 取名, 穿衣, 搭配, 颜色, 五行色, 开运色, 今日运势, 每日运程, 合婚, 择吉, 生命灵数, 风水, 算命, BaZi, ZiWei, QiMen, Tarot, feng shui, I Ching, numerology, daily horoscope, palmistry, physiognomy, naming, dressing
|
||||
# 触发关键词收紧:1.1.8 移除「算命 / 占卜 / 命理 / 数字命理 / fortune telling / astrology」等过宽泛词,
|
||||
# 避免在用户未明确要求占卜的对话中误激活本 skill;仅在明确出现具体体系名称或具体主题时才被触发。
|
||||
openclaw:
|
||||
emoji: "☯️"
|
||||
skillKey: "university-applications"
|
||||
runtime:
|
||||
node: ">=18"
|
||||
python3: true
|
||||
install:
|
||||
- kind: node
|
||||
package: iztro
|
||||
env: [] # 本 skill 不再读取任何环境变量;曾经的 OPENCLAW_KNOWLEDGE_DIR 已被移除以避免 file-system enumeration。
|
||||
security:
|
||||
network:
|
||||
default: none
|
||||
optional:
|
||||
- feature: "liuyao HTML LLM divination (user-initiated, in-browser, OFF by default)"
|
||||
allowed-endpoints:
|
||||
- "https://api.openai.com"
|
||||
- "https://api.anthropic.com"
|
||||
- "https://api.deepseek.com"
|
||||
custom-endpoint: "only after explicit in-UI consent dialog; HTTPS-only, no IP/localhost auto-trust"
|
||||
data-sent: "only the hexagram and the user's typed question; no profile, no env vars, no file paths"
|
||||
credential: "user-provided LLM API key, entered at runtime, stored in browser localStorage only"
|
||||
- feature: "liuyao HTML Google Fonts (commented out by default)"
|
||||
endpoint: "https://fonts.googleapis.com"
|
||||
data-sent: "none beyond standard font request"
|
||||
credential: none
|
||||
credentials:
|
||||
bundled: none
|
||||
required: none
|
||||
user-optional:
|
||||
- "LLM API key for liuyao/index.html divination feature (scope it to a separate limited key, never reused)"
|
||||
push-mechanism: openclaw-ipc
|
||||
push-optin: true
|
||||
push-default-state: disabled
|
||||
data-retention:
|
||||
location: "local filesystem under data/profiles/ and data/push-log.json"
|
||||
remote-upload: none
|
||||
user-controls:
|
||||
- "view: node scripts/profile.js show <userId>"
|
||||
- "list: node scripts/profile.js list"
|
||||
- "edit: node scripts/profile.js save <userId> <field> <value>"
|
||||
- "delete: node scripts/profile.js delete <userId>"
|
||||
- "disable push: node scripts/push-toggle.js off <userId>"
|
||||
notes: |
|
||||
All bundled scripts perform local computation only — no fetch, axios,
|
||||
https.request, curl, wget, or any outbound network calls from the Node/Python
|
||||
side. Push delivery is handled entirely by the OpenClaw runtime via stdout/IPC
|
||||
protocol, and is OPT-IN (disabled until the user runs push-toggle.js on).
|
||||
The 'channels' field in user profiles (e.g. telegram) is a routing hint for
|
||||
the OpenClaw runtime, not a direct API integration. This skill does not hold
|
||||
or require any third-party API tokens (Telegram Bot Token, SMTP credentials,
|
||||
webhook URLs, etc.). The local-only release helper script is excluded
|
||||
from the published bundle via .clawhubignore and is not part of the
|
||||
installed skill surface.
|
||||
EXCEPTION — OPTIONAL LLM NETWORK USE: the browser-only file liuyao/index.html
|
||||
exposes an optional "LLM divination" button. If and only if the user clicks
|
||||
it and fills in their own API key + endpoint, the browser (not the skill
|
||||
process) will POST the hexagram and question to that user-configured endpoint.
|
||||
No key is bundled, hardcoded, or transmitted anywhere else. Users are advised
|
||||
to supply a scoped/limited API key rather than a primary account key.
|
||||
User profile data (birth details, optional family members, interaction log)
|
||||
is stored only on the local filesystem and can be viewed, edited, or deleted
|
||||
at any time via scripts/profile.js (see data-retention.user-controls above).
|
||||
---
|
||||
|
||||
# body
|
||||
Reference in New Issue
Block a user