WeKnora deployment source with the reviewed wiki identity, summary and recovery fixes
Anydoc / anydoc parser engine (push) Canceled after 0s
App / Format, vet, test, and build (push) Canceled after 0s
CLI / test (macos-latest / go 1.26) (push) Canceled after 0s
CLI / test (ubuntu-latest / go 1.26) (push) Canceled after 0s
CLI / test (windows-latest / go 1.26) (push) Canceled after 0s
Build and Push Docker Image / build-ui (push) Canceled after 0s
Build and Push Docker Image / build-docreader (push) Canceled after 0s
Build and Push Docker Image / build-sandbox (push) Canceled after 0s
Build and Push Docker Image / build-app (amd64, linux/amd64, ubuntu-latest) (push) Canceled after 0s
Build and Push Docker Image / build-app (arm64, linux/arm64, ubuntu-24.04-arm) (push) Canceled after 0s
DocReader / Python tests and Go client tests (push) Canceled after 0s
Build & Publish (dsh-weknora) / Typecheck, test, pack (push) Canceled after 0s
Build & Publish (dsh-weknora) / End-to-end inside dsh (pinned) (push) Canceled after 0s
Build & Publish (dsh-weknora) / End-to-end inside dsh (latest, advisory) (push) Canceled after 0s
Frontend / Test, type-check, and build (push) Canceled after 0s
Go Lint Cache / warm lint cache (.) (push) Canceled after 0s
Go Lint Cache / warm lint cache (cli) (push) Canceled after 0s
Go Lint Cache / warm lint cache (client) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.10) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.11) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.12) (push) Canceled after 0s
Build & Publish (mcp-server) / Test (Python 3.13) (push) Canceled after 0s
Build and Push Docker Image / merge (push) Canceled after 0s
Build & Publish (dsh-weknora) / Publish to npm (push) Canceled after 0s
Build & Publish (mcp-server) / Build sdist + wheel (push) Canceled after 0s
Build & Publish (mcp-server) / Publish to PyPI (push) Canceled after 0s

This commit is contained in:
yawan
2026-09-22 16:19:06 +08:00
commit f7fc992f57
3929 changed files with 1006607 additions and 0 deletions
+92
View File
@@ -0,0 +1,92 @@
package skills
import (
"context"
"fmt"
"strings"
"github.com/Tencent/WeKnora/internal/sandbox"
)
// SkillEnvResolver produces the environment one execution gets. It is separate
// from SkillSource because the values are per-caller: the same skill, in the
// same image, hands a different key to a different Principal. The implementation
// lives in the service layer, which can reach the repository.
//
// It is keyed by skill NAME rather than id because every path into the manager
// is name-addressed: ExecuteScript receives the name the model wrote, and the
// row id is an implementation detail of the installed-skill source.
type SkillEnvResolver interface {
// ResolveEnv returns the values to inject and the names of any required
// variable that neither the admin nor this caller has filled in. An empty
// skillName asks for the caller's config-wide variables alone. The caller's
// identity is taken from ctx, never from a parameter.
ResolveEnv(ctx context.Context, skillName string) (env map[string]string, missing []string, err error)
}
// MissingSkillEnvError reports that execution was refused because a required
// variable has no value at either layer. It is typed so the agent loop can
// relay a sentence a person can act on instead of the KeyError or 401 the
// script would otherwise produce.
type MissingSkillEnvError struct {
SkillName string
Names []string
}
func (e *MissingSkillEnvError) Error() string {
// English, like every other error in this codebase: the agent relays this
// to the user and translates it into whatever language they are speaking.
// execute_skill_script has no env parameter, so it cannot take a value the
// user just typed. shell_exec can: naming the skill and passing the value
// in env runs the command and stores the value for the next run. Pointing
// at the settings page alone would strand IM users, who have no such page.
return fmt.Sprintf(
"skill %q needs the environment variable(s) %s, which nobody has set yet. "+
"Ask the user for them, then run the skill through shell_exec with "+
"skill_name=%q and the values in env — they are stored for that user "+
"afterwards. They can also be set under Settings → Sandbox secrets.",
e.SkillName, strings.Join(e.Names, ", "), e.SkillName,
)
}
// ApplyResolvedEnv overlays resolved onto env WITHOUT displacing anything env
// already carries. It is exported because shell_exec applies the same overlay
// for its optional skill_name parameter and must not diverge from it.
//
// This is the second layer of reserved-name protection. Task 2's write-time
// blacklist is the first, but a value written before that blacklist existed
// would still be in the database, and letting it land on
// WEKNORA_SKILL_OUTPUT_DIR would silently redirect the turn's artifacts to a
// directory nobody drains. Skipping existing keys makes that impossible
// regardless of what is stored.
func ApplyResolvedEnv(env, resolved map[string]string) {
for name, value := range resolved {
if _, taken := env[name]; taken {
continue
}
env[name] = value
}
}
// applySessionPackagePath prepends the per-session extra-packages directory
// so a frozen skill venv can still see `pip install --target` extras without
// mutating the snapshot. Missing directories are ignored by the interpreters.
func applySessionPackagePath(env map[string]string, skillName string) {
if env == nil {
return
}
dir := sandbox.SessionSkillPackageDir(skillName)
if dir == "" {
return
}
prependPathEnv(env, pythonPathEnvVar, dir)
prependPathEnv(env, nodePathEnvVar, dir)
}
func prependPathEnv(env map[string]string, key, dir string) {
if existing := strings.TrimSpace(env[key]); existing != "" {
env[key] = dir + ":" + existing
return
}
env[key] = dir
}
+151
View File
@@ -0,0 +1,151 @@
package skills
import (
"context"
"errors"
"testing"
"github.com/Tencent/WeKnora/internal/sandbox"
"github.com/stretchr/testify/require"
)
// The runtime seeds the artifact directory variables before the resolver runs.
// Whatever a user stored, those keys must survive untouched: they are how the
// turn's products are collected.
func TestApplyResolvedEnvAddsNewKeysAndKeepsExistingOnes(t *testing.T) {
env := map[string]string{
artifactOutputEnvVar: "/workspace/output",
artifactHistoryEnvVar: "/workspace/output",
}
ApplyResolvedEnv(env, map[string]string{
"TAVILY_API_KEY": "user-key",
artifactOutputEnvVar: "/tmp/hijacked",
artifactHistoryEnvVar: "/tmp/hijacked",
})
require.Equal(t, "user-key", env["TAVILY_API_KEY"])
require.Equal(t, "/workspace/output", env[artifactOutputEnvVar])
require.Equal(t, "/workspace/output", env[artifactHistoryEnvVar])
}
func TestApplySessionPackagePathPrependsPythonAndNodePath(t *testing.T) {
env := map[string]string{pythonPathEnvVar: "/already"}
applySessionPackagePath(env, "律师助手")
dir := sandbox.SessionSkillPackageDir("律师助手")
require.Equal(t, dir+":/already", env[pythonPathEnvVar])
require.Equal(t, dir, env[nodePathEnvVar])
}
// The message is read by the agent and relayed verbatim to a person, so it has
// to name the skill, the variables, and where to go and fix it.
func TestMissingSkillEnvErrorMessageNamesSkillVarsAndWhereToSetThem(t *testing.T) {
err := error(&MissingSkillEnvError{
SkillName: "web-search",
Names: []string{"TAVILY_API_KEY", "SERP_TOKEN"},
})
msg := err.Error()
require.Contains(t, msg, "web-search")
require.Contains(t, msg, "TAVILY_API_KEY")
require.Contains(t, msg, "SERP_TOKEN")
require.Contains(t, msg, "Sandbox secrets")
var typed *MissingSkillEnvError
require.True(t, errors.As(err, &typed))
require.Equal(t, "web-search", typed.SkillName)
require.Equal(t, []string{"TAVILY_API_KEY", "SERP_TOKEN"}, typed.Names)
}
// stubEnvResolver records what ExecuteScript asked it for.
type stubEnvResolver struct {
env map[string]string
missing []string
err error
calls []string
}
func (r *stubEnvResolver) ResolveEnv(
_ context.Context, skillName string,
) (map[string]string, []string, error) {
r.calls = append(r.calls, skillName)
return r.env, r.missing, r.err
}
func TestExecuteScriptBlocksOnMissingRequiredEnv(t *testing.T) {
dir := preloadedSkillDir(t, "web-search", "search the web")
sandboxMgr := &recordingSandboxManager{}
resolver := &stubEnvResolver{missing: []string{"TAVILY_API_KEY"}}
mgr := NewManager(&ManagerConfig{SkillDirs: []string{dir}, Enabled: true}, sandboxMgr).
WithEnvResolver(resolver)
require.NoError(t, mgr.Initialize(context.Background()))
result, err := mgr.ExecuteScript(context.Background(), "web-search", "scripts/run.py", nil, "")
require.Nil(t, result)
var missing *MissingSkillEnvError
require.ErrorAs(t, err, &missing)
require.Equal(t, "web-search", missing.SkillName)
require.Equal(t, []string{"TAVILY_API_KEY"}, missing.Names)
// The whole point of blocking is that the script never runs.
require.Zero(t, sandboxMgr.calls)
require.Nil(t, sandboxMgr.config)
}
func TestExecuteScriptPropagatesResolverError(t *testing.T) {
dir := preloadedSkillDir(t, "web-search", "search the web")
sandboxMgr := &recordingSandboxManager{}
boom := errors.New("load user envs: connection refused")
mgr := NewManager(&ManagerConfig{SkillDirs: []string{dir}, Enabled: true}, sandboxMgr).
WithEnvResolver(&stubEnvResolver{err: boom})
require.NoError(t, mgr.Initialize(context.Background()))
_, err := mgr.ExecuteScript(context.Background(), "web-search", "scripts/run.py", nil, "")
require.ErrorIs(t, err, boom)
require.Zero(t, sandboxMgr.calls)
}
func TestExecuteScriptInjectsResolvedEnvOnceKeyedBySkillName(t *testing.T) {
dir := preloadedSkillDir(t, "web-search", "search the web")
sandboxMgr := &recordingSandboxManager{}
resolver := &stubEnvResolver{env: map[string]string{
"TAVILY_API_KEY": "user-key",
artifactOutputEnvVar: "/tmp/hijacked",
}}
mgr := NewManager(&ManagerConfig{SkillDirs: []string{dir}, Enabled: true}, sandboxMgr).
WithEnvResolver(resolver)
require.NoError(t, mgr.Initialize(context.Background()))
_, err := mgr.ExecuteScript(context.Background(), "web-search", "scripts/run.py", nil, "")
require.NoError(t, err)
require.Equal(t, []string{"web-search"}, resolver.calls)
require.Equal(t, 1, sandboxMgr.calls)
require.NotNil(t, sandboxMgr.config)
require.Equal(t, "user-key", sandboxMgr.config.Env["TAVILY_API_KEY"])
require.Equal(t, "/workspace/output", sandboxMgr.config.Env[artifactOutputEnvVar])
}
// Without a resolver the execution path must stay exactly as it was.
func TestExecuteScriptWithoutResolverInjectsNothingExtra(t *testing.T) {
dir := preloadedSkillDir(t, "web-search", "search the web")
sandboxMgr := &recordingSandboxManager{}
mgr := NewManager(&ManagerConfig{SkillDirs: []string{dir}, Enabled: true}, sandboxMgr)
require.NoError(t, mgr.Initialize(context.Background()))
_, err := mgr.ExecuteScript(context.Background(), "web-search", "scripts/run.py", nil, "")
require.NoError(t, err)
require.Equal(t, 1, sandboxMgr.calls)
injected := map[string]bool{}
for _, name := range InjectedSandboxEnvVars() {
injected[name] = true
}
for key := range sandboxMgr.config.Env {
require.True(t, injected[key], "injected env %q is not in InjectedSandboxEnvVars", key)
}
}
+161
View File
@@ -0,0 +1,161 @@
package skills
import (
"context"
"path/filepath"
"runtime"
"testing"
)
// TestExampleSkillsIntegration tests with the actual example skills in examples/skills
func TestExampleSkillsIntegration(t *testing.T) {
// Get the path to examples/skills relative to this test file
_, filename, _, ok := runtime.Caller(0)
if !ok {
t.Fatal("Failed to get current file path")
}
// Navigate from internal/agent/skills to examples/skills
skillsDir := filepath.Join(filepath.Dir(filename), "..", "..", "..", "examples", "skills")
// Create loader
loader := NewLoader([]string{skillsDir})
// Discover skills
metadata, err := loader.DiscoverSkills()
if err != nil {
t.Fatalf("Failed to discover skills: %v", err)
}
if len(metadata) == 0 {
t.Skip("No example skills found in examples/skills directory")
}
t.Logf("Discovered %d example skills:", len(metadata))
for _, m := range metadata {
t.Logf(" - %s: %s", m.Name, truncate(m.Description, 60))
}
// Test loading the pdf-processing skill
pdfSkill, err := loader.LoadSkillInstructions("pdf-processing")
if err != nil {
t.Fatalf("Failed to load pdf-processing skill: %v", err)
}
// Verify metadata
if pdfSkill.Name != "pdf-processing" {
t.Errorf("Expected name 'pdf-processing', got '%s'", pdfSkill.Name)
}
if pdfSkill.Instructions == "" {
t.Error("Expected instructions to be non-empty")
}
t.Logf("PDF Processing skill instructions length: %d characters", len(pdfSkill.Instructions))
// Test loading additional file (FORMS.md)
formsFile, err := loader.LoadSkillFile("pdf-processing", "FORMS.md")
if err != nil {
t.Fatalf("Failed to load FORMS.md: %v", err)
}
if formsFile.Content == "" {
t.Error("Expected FORMS.md content to be non-empty")
}
t.Logf("FORMS.md content length: %d characters", len(formsFile.Content))
// Test loading script
scriptFile, err := loader.LoadSkillFile("pdf-processing", "scripts/analyze_form.py")
if err != nil {
t.Fatalf("Failed to load analyze_form.py: %v", err)
}
if !scriptFile.IsScript {
t.Error("analyze_form.py should be marked as script")
}
t.Logf("analyze_form.py content length: %d characters", len(scriptFile.Content))
// Test list files
files, err := loader.ListSkillFiles("pdf-processing")
if err != nil {
t.Fatalf("Failed to list skill files: %v", err)
}
t.Logf("Files in pdf-processing skill:")
for _, f := range files {
t.Logf(" - %s (script: %v)", f, IsScript(f))
}
}
// TestManagerWithExampleSkills tests the Manager with example skills
func TestManagerWithExampleSkills(t *testing.T) {
// Get the path to examples/skills
_, filename, _, ok := runtime.Caller(0)
if !ok {
t.Fatal("Failed to get current file path")
}
skillsDir := filepath.Join(filepath.Dir(filename), "..", "..", "..", "examples", "skills")
// Create manager
config := &ManagerConfig{
SkillDirs: []string{skillsDir},
AllowedSkills: []string{}, // Allow all
Enabled: true,
}
manager := NewManager(config, nil)
// Initialize
ctx := context.Background()
if err := manager.Initialize(ctx); err != nil {
t.Fatalf("Failed to initialize manager: %v", err)
}
// Get metadata for system prompt
metadata := manager.GetAllMetadata()
if len(metadata) == 0 {
t.Skip("No example skills found")
}
t.Logf("Manager discovered %d skills for system prompt injection", len(metadata))
// Simulate what the agent would do:
// 1. First, get metadata (Level 1 - already in system prompt)
for _, m := range metadata {
t.Logf("Level 1 (metadata): %s - %s", m.Name, truncate(m.Description, 50))
}
// 2. When user request matches, load full skill instructions (Level 2)
skill, err := manager.LoadSkill(ctx, "pdf-processing")
if err != nil {
t.Fatalf("Failed to load skill: %v", err)
}
t.Logf("Level 2 (instructions): Loaded %d characters of instructions", len(skill.Instructions))
// 3. If skill references additional files, read them (Level 3)
formsContent, err := manager.ReadSkillFile(ctx, "pdf-processing", "FORMS.md")
if err != nil {
t.Fatalf("Failed to read skill file: %v", err)
}
t.Logf("Level 3 (resources): Loaded FORMS.md with %d characters", len(formsContent))
// Test GetSkillInfo
info, err := manager.GetSkillInfo(ctx, "pdf-processing")
if err != nil {
t.Fatalf("Failed to get skill info: %v", err)
}
t.Logf("Skill info: name=%s, files=%d", info.Name, len(info.Files))
}
func truncate(s string, maxLen int) string {
if len(s) <= maxLen {
return s
}
return s[:maxLen] + "..."
}
+308
View File
@@ -0,0 +1,308 @@
package skills
import (
"fmt"
"os"
"path/filepath"
"strings"
)
// Loader handles skill discovery and loading from the filesystem
// It implements the Progressive Disclosure pattern by separating
// metadata discovery (Level 1) from instructions loading (Level 2/3)
type Loader struct {
// skillDirs are the directories to search for skills
skillDirs []string
// discoveredSkills caches discovered skill metadata
discoveredSkills map[string]*Skill
}
// NewLoader creates a new skill loader with the specified search directories
func NewLoader(skillDirs []string) *Loader {
return &Loader{
skillDirs: skillDirs,
discoveredSkills: make(map[string]*Skill),
}
}
// DiscoverSkills scans all configured directories for SKILL.md files
// and extracts their metadata (Level 1). This is a lightweight operation
// that only reads the frontmatter of each skill file.
func (l *Loader) DiscoverSkills() ([]*SkillMetadata, error) {
var allMetadata []*SkillMetadata
for _, dir := range l.skillDirs {
metadata, err := l.discoverInDirectory(dir)
if err != nil {
// Log warning but continue with other directories
continue
}
allMetadata = append(allMetadata, metadata...)
}
return allMetadata, nil
}
// discoverInDirectory scans a single directory for skill subdirectories
func (l *Loader) discoverInDirectory(dir string) ([]*SkillMetadata, error) {
var metadata []*SkillMetadata
// Check if directory exists
info, err := os.Stat(dir)
if err != nil {
if os.IsNotExist(err) {
return nil, nil // Directory doesn't exist, skip silently
}
return nil, fmt.Errorf("failed to access skill directory %s: %w", dir, err)
}
if !info.IsDir() {
return nil, fmt.Errorf("%s is not a directory", dir)
}
// Read directory entries
entries, err := os.ReadDir(dir)
if err != nil {
return nil, fmt.Errorf("failed to read skill directory %s: %w", dir, err)
}
for _, entry := range entries {
if !entry.IsDir() {
continue
}
skillPath := filepath.Join(dir, entry.Name())
skillFile := filepath.Join(skillPath, SkillFileName)
// Check if SKILL.md exists
if _, err := os.Stat(skillFile); os.IsNotExist(err) {
continue
}
// Read and parse SKILL.md
content, err := os.ReadFile(skillFile)
if err != nil {
continue
}
skill, err := ParseSkillFile(string(content))
if err != nil {
continue
}
// Set filesystem paths
skill.BasePath = skillPath
skill.FilePath = skillFile
// Cache the skill
l.discoveredSkills[skill.Name] = skill
metadata = append(metadata, skill.ToMetadata())
}
return metadata, nil
}
// LoadSkillInstructions loads the full instructions of a skill (Level 2)
// Returns the cached skill if already loaded
func (l *Loader) LoadSkillInstructions(skillName string) (*Skill, error) {
// Check cache first
if skill, ok := l.discoveredSkills[skillName]; ok {
if skill.Loaded {
return skill, nil
}
}
// Search for the skill in all directories
for _, dir := range l.skillDirs {
skill, err := l.loadSkillFromDirectory(dir, skillName)
if err == nil {
l.discoveredSkills[skillName] = skill
return skill, nil
}
}
return nil, fmt.Errorf("skill not found: %s", skillName)
}
// loadSkillFromDirectory attempts to load a skill from a specific directory
func (l *Loader) loadSkillFromDirectory(dir, skillName string) (*Skill, error) {
// First, check if we can find by directory name matching skill name
skillPath := filepath.Join(dir, skillName)
skillFile := filepath.Join(skillPath, SkillFileName)
if _, err := os.Stat(skillFile); err == nil {
return l.loadSkillFile(skillPath, skillFile)
}
// Otherwise, scan all subdirectories to find the skill by name
entries, err := os.ReadDir(dir)
if err != nil {
return nil, err
}
for _, entry := range entries {
if !entry.IsDir() {
continue
}
skillPath := filepath.Join(dir, entry.Name())
skillFile := filepath.Join(skillPath, SkillFileName)
if _, err := os.Stat(skillFile); os.IsNotExist(err) {
continue
}
content, err := os.ReadFile(skillFile)
if err != nil {
continue
}
skill, err := ParseSkillFile(string(content))
if err != nil {
continue
}
if skill.Name == skillName {
skill.BasePath = skillPath
skill.FilePath = skillFile
return skill, nil
}
}
return nil, fmt.Errorf("skill not found in %s: %s", dir, skillName)
}
// loadSkillFile reads and parses a SKILL.md file
func (l *Loader) loadSkillFile(basePath, filePath string) (*Skill, error) {
content, err := os.ReadFile(filePath)
if err != nil {
return nil, fmt.Errorf("failed to read skill file: %w", err)
}
skill, err := ParseSkillFile(string(content))
if err != nil {
return nil, err
}
skill.BasePath = basePath
skill.FilePath = filePath
return skill, nil
}
// LoadSkillFile loads an additional file from a skill directory (Level 3)
// The filePath should be relative to the skill's base directory
func (l *Loader) LoadSkillFile(skillName, relativePath string) (*SkillFile, error) {
// Get the skill first
skill, ok := l.discoveredSkills[skillName]
if !ok {
// Try to load the skill
var err error
skill, err = l.LoadSkillInstructions(skillName)
if err != nil {
return nil, fmt.Errorf("skill not found: %s", skillName)
}
}
// Validate and resolve the file path
cleanPath := filepath.Clean(relativePath)
// Security: prevent path traversal
if strings.HasPrefix(cleanPath, "..") || filepath.IsAbs(cleanPath) {
return nil, fmt.Errorf("invalid file path: %s", relativePath)
}
fullPath := filepath.Join(skill.BasePath, cleanPath)
// Verify the file is within the skill directory
absSkillPath, err := filepath.Abs(skill.BasePath)
if err != nil {
return nil, err
}
absFilePath, err := filepath.Abs(fullPath)
if err != nil {
return nil, err
}
if !strings.HasPrefix(absFilePath, absSkillPath) {
return nil, fmt.Errorf("file path outside skill directory: %s", relativePath)
}
// Read the file
content, err := os.ReadFile(fullPath)
if err != nil {
return nil, fmt.Errorf("failed to read file: %w", err)
}
return &SkillFile{
Name: relativePath,
Path: absFilePath, // Use absolute path for sandbox execution
Content: string(content),
IsScript: IsScript(relativePath),
}, nil
}
// ListSkillFiles lists all files in a skill directory
func (l *Loader) ListSkillFiles(skillName string) ([]string, error) {
skill, ok := l.discoveredSkills[skillName]
if !ok {
var err error
skill, err = l.LoadSkillInstructions(skillName)
if err != nil {
return nil, fmt.Errorf("skill not found: %s", skillName)
}
}
var files []string
err := filepath.Walk(skill.BasePath, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if info.IsDir() {
return nil
}
// Get relative path
relPath, err := filepath.Rel(skill.BasePath, path)
if err != nil {
return err
}
files = append(files, relPath)
return nil
})
if err != nil {
return nil, fmt.Errorf("failed to list skill files: %w", err)
}
return files, nil
}
// GetSkillByName returns a cached skill by name
func (l *Loader) GetSkillByName(name string) (*Skill, bool) {
skill, ok := l.discoveredSkills[name]
return skill, ok
}
// GetSkillBasePath returns the base path for a skill (always absolute)
func (l *Loader) GetSkillBasePath(skillName string) (string, error) {
skill, ok := l.discoveredSkills[skillName]
if !ok {
var err error
skill, err = l.LoadSkillInstructions(skillName)
if err != nil {
return "", fmt.Errorf("skill not found: %s", skillName)
}
}
// Return absolute path for consistent sandbox execution
return filepath.Abs(skill.BasePath)
}
// Reload clears the cache and rediscovers all skills
func (l *Loader) Reload() ([]*SkillMetadata, error) {
l.discoveredSkills = make(map[string]*Skill)
return l.DiscoverSkills()
}
+601
View File
@@ -0,0 +1,601 @@
package skills
import (
"context"
"fmt"
"os"
"strings"
"sync"
"github.com/Tencent/WeKnora/internal/logger"
"github.com/Tencent/WeKnora/internal/sandbox"
"github.com/Tencent/WeKnora/internal/types"
)
// artifactOutputEnvVar is the name of the environment variable that WeKnora
// injects into every skill script execution. The value points to the
// convention-driven directory where the script should drop artifacts the user
// will be able to download after the turn completes.
//
// The name is stable across releases; skills reference it via os.getenv(...)
// so they never hard-code the path.
const artifactOutputEnvVar = "WEKNORA_SKILL_OUTPUT_DIR"
// sessionInputEnvVar points skill scripts at user-uploaded files restored into
// the current session's Cube. Inputs are separate from generated artifacts.
const sessionInputEnvVar = "WEKNORA_SESSION_INPUT_DIR"
// artifactHistoryEnvVar is the name of the environment variable that points
// to the root artifact output directory (/workspace/output). Skill scripts
// can use this to self-discover artifacts from prior runs when they need to
// chain without LLM mediation.
const artifactHistoryEnvVar = "WEKNORA_SKILL_HISTORY_ROOT"
// skillDirEnvVar points a script at its own directory inside the sandbox
// image. Installed skills run with /workspace as WorkDir, so this is how a
// script reaches the data and helpers that were installed beside it. The
// install-time verification pass exports the same name.
const skillDirEnvVar = "WEKNORA_SKILL_DIR"
// pythonPathEnvVar / nodePathEnvVar carry the per-session extra-packages
// overlay (see sandbox.SessionSkillPackageDir). They are injected rather than
// left for the skill to declare so a stored PYTHONPATH cannot displace them.
const pythonPathEnvVar = "PYTHONPATH"
const nodePathEnvVar = "NODE_PATH"
// InjectedSandboxEnvVars is every name ExecuteScript writes into the sandbox
// environment. The skill-env declaration blacklist must reject these so a
// stored value cannot redirect artifacts, the skill directory, or the session
// input tree. Credential names such as WEKNORA_API_KEY are not in this list.
func InjectedSandboxEnvVars() []string {
return []string{
artifactOutputEnvVar,
sessionInputEnvVar,
artifactHistoryEnvVar,
skillDirEnvVar,
pythonPathEnvVar,
nodePathEnvVar,
}
}
// defaultArtifactOutputDir is used when neither the environment variable
// (WEKNORA_SKILL_OUTPUT_DIR) nor the ExecuteConfig.Env has an override.
// /workspace/output sits inside the base sandbox image's writable tree and
// is guaranteed to survive across Execute calls for the same session (Cube
// SessionBoundManager keeps the MicroVM alive between calls).
const defaultArtifactOutputDir = "/workspace/output"
// ArtifactOutputDir returns the absolute path (inside the sandbox) where
// skill scripts should write artifacts for this turn. It is exported so
// callers such as ArtifactCollector can list the same directory when
// draining artifacts after Execute returns.
//
// Resolution order (first usable wins):
// 1. WEKNORA_SKILL_OUTPUT_DIR from the host environment (ops override), when
// it names a directory inside the session workspace.
// 2. defaultArtifactOutputDir.
//
// The override goes through sandbox.ValidatedSessionOutputDir, the same gate
// the sandbox applies to a tenant's override. An operator who points this
// outside /workspace would otherwise send the readers (this function feeds the
// sandbox file tools and ArtifactCollector) to a directory no skill can write,
// since execution refuses the same path and falls back.
//
// Callers are expected to treat the returned string as read-only: the path
// is normalised (no trailing slash) so it can be joined safely.
func ArtifactOutputDir() string {
if v := strings.TrimSpace(os.Getenv(artifactOutputEnvVar)); v != "" {
if clean, ok := sandbox.ValidatedSessionOutputDir(v); ok {
return clean
}
}
return defaultArtifactOutputDir
}
// SkillOutputDir returns the artifact output directory for skill executions.
// All skills write to the same root directory (/workspace/output/) to enable
// collaboration and file sharing between different skill executions.
func (m *Manager) SkillOutputDir(sessionID, skillName string) string {
return ArtifactOutputDir()
}
// Manager manages skills lifecycle including discovery, loading, and script execution
// It coordinates between the Loader (filesystem operations) and Sandbox (script execution)
type Manager struct {
loader *Loader
sandboxMgr sandbox.Manager
// tenantSource holds the skills installed into this run's sandbox image.
// When set it is the only source the model is told about: the host
// skills/preloaded directory is not what execute_skill_script would find
// inside the sandbox.
tenantSource SkillSource
// envResolver supplies the per-caller environment for one execution. It
// is nil when the run has no installed skills, in which case execution
// keeps exactly its previous behaviour.
envResolver SkillEnvResolver
// Configuration
skillDirs []string
allowedSkills []string // Empty means all skills are allowed
enabled bool
// Cache
metadataCache []*SkillMetadata
mu sync.RWMutex
}
// ManagerConfig holds configuration for the skill manager
type ManagerConfig struct {
SkillDirs []string // Directories to search for skills
AllowedSkills []string // Skill names whitelist (empty = allow all)
Enabled bool // Whether skills are enabled
}
// NewManager creates a new skill manager with the given configuration
func NewManager(config *ManagerConfig, sandboxMgr sandbox.Manager) *Manager {
if config == nil {
config = &ManagerConfig{
Enabled: false,
}
}
return &Manager{
loader: NewLoader(config.SkillDirs),
sandboxMgr: sandboxMgr,
skillDirs: config.SkillDirs,
allowedSkills: config.AllowedSkills,
enabled: config.Enabled,
}
}
// IsEnabled returns whether skills are enabled
func (m *Manager) IsEnabled() bool {
return m.enabled
}
// WithTenantSource attaches the skills an administrator installed into the
// sandbox config this run booted from. It is part of construction - callers
// must invoke it before Initialize, i.e. before the engine can reach the
// manager - so it takes no lock.
func (m *Manager) WithTenantSource(source SkillSource) *Manager {
m.tenantSource = source
return m
}
// WithEnvResolver attaches the per-caller environment resolver. Like
// WithTenantSource it is part of construction and must be invoked before
// Initialize, so it takes no lock.
func (m *Manager) WithEnvResolver(resolver SkillEnvResolver) *Manager {
m.envResolver = resolver
return m
}
// resolveSource decides which source owns one skill name. An installed image
// is the only copy the sandbox can run: falling back to a host preloaded
// skill would advertise files that are not in the image.
func (m *Manager) resolveSource(skillName string) SkillSource {
if m.tenantSource != nil {
return m.tenantSource
}
return m.loader
}
// discoverAllSkills returns the set the model is told about. When skills are
// installed into the sandbox image, that image is the source of truth; the
// deployment's skills/preloaded directory is not what execute_skill_script
// would find inside the sandbox.
func (m *Manager) discoverAllSkills() ([]*SkillMetadata, error) {
if m.tenantSource != nil {
return m.tenantSource.DiscoverSkills()
}
return m.loader.Reload()
}
// Initialize discovers all skills and caches their metadata
// This should be called at startup
func (m *Manager) Initialize(ctx context.Context) error {
if !m.enabled {
return nil
}
metadata, err := m.discoverAllSkills()
if err != nil {
return fmt.Errorf("failed to discover skills: %w", err)
}
// Filter by allowed skills if specified
if len(m.allowedSkills) > 0 {
metadata = m.filterAllowedSkills(metadata)
}
m.mu.Lock()
m.metadataCache = metadata
m.mu.Unlock()
return nil
}
// filterAllowedSkills filters metadata to only include allowed skills
func (m *Manager) filterAllowedSkills(metadata []*SkillMetadata) []*SkillMetadata {
if len(m.allowedSkills) == 0 {
return metadata
}
allowedSet := make(map[string]bool)
for _, name := range m.allowedSkills {
allowedSet[name] = true
}
var filtered []*SkillMetadata
for _, meta := range metadata {
if allowedSet[meta.Name] {
filtered = append(filtered, meta)
}
}
return filtered
}
// GetAllMetadata returns metadata for all discovered skills
// This is used for system prompt injection (Level 1)
func (m *Manager) GetAllMetadata() []*SkillMetadata {
if !m.enabled {
return nil
}
m.mu.RLock()
defer m.mu.RUnlock()
// Return a copy to prevent external modification
result := make([]*SkillMetadata, len(m.metadataCache))
copy(result, m.metadataCache)
return result
}
// LoadSkill loads the full instructions of a skill (Level 2)
func (m *Manager) LoadSkill(ctx context.Context, skillName string) (*Skill, error) {
if !m.enabled {
return nil, fmt.Errorf("skills are not enabled")
}
// Check if skill is allowed
if !m.isSkillAllowed(skillName) {
return nil, fmt.Errorf("skill not allowed: %s", skillName)
}
return m.resolveSource(skillName).LoadSkillInstructions(skillName)
}
// isSkillAllowed checks if a skill is in the allowed list
func (m *Manager) isSkillAllowed(skillName string) bool {
if len(m.allowedSkills) == 0 {
return true
}
for _, name := range m.allowedSkills {
if name == skillName {
return true
}
}
return false
}
// ReadSkillFile reads an additional file from a skill directory (Level 3)
func (m *Manager) ReadSkillFile(ctx context.Context, skillName, filePath string) (string, error) {
if !m.enabled {
return "", fmt.Errorf("skills are not enabled")
}
if !m.isSkillAllowed(skillName) {
return "", fmt.Errorf("skill not allowed: %s", skillName)
}
file, err := m.resolveSource(skillName).LoadSkillFile(skillName, filePath)
if err != nil {
return "", err
}
return file.Content, nil
}
// ListSkillFiles lists all files in a skill directory
func (m *Manager) ListSkillFiles(ctx context.Context, skillName string) ([]string, error) {
if !m.enabled {
return nil, fmt.Errorf("skills are not enabled")
}
if !m.isSkillAllowed(skillName) {
return nil, fmt.Errorf("skill not allowed: %s", skillName)
}
return m.resolveSource(skillName).ListSkillFiles(skillName)
}
// SandboxSkillDir reports where a skill lives inside the sandbox image, and
// whether that path means anything to say out loud.
//
// Only an installed skill has one. A preloaded skill is uploaded from the host
// for the duration of a single call, so its base path names a directory on the
// WeKnora machine that no shell command in the sandbox can reach — telling the
// model about it would be worse than saying nothing.
func (m *Manager) SandboxSkillDir(skillName string) (string, bool) {
if m == nil || !m.enabled || !m.isSkillAllowed(skillName) {
return "", false
}
image, ok := m.resolveSource(skillName).(imageSkillSource)
if !ok {
return "", false
}
dir, err := image.GetSkillBasePath(skillName)
if err != nil {
return "", false
}
dir = strings.TrimSpace(dir)
return dir, dir != ""
}
// ExecuteScript executes a script from a skill in the sandbox
func (m *Manager) ExecuteScript(ctx context.Context, skillName, scriptPath string, args []string, stdin string) (*sandbox.ExecuteResult, error) {
if !m.enabled {
return nil, fmt.Errorf("skills are not enabled")
}
if !m.isSkillAllowed(skillName) {
return nil, fmt.Errorf("skill not allowed: %s", skillName)
}
// Verify sandbox manager is available
if m.sandboxMgr == nil {
return nil, fmt.Errorf("sandbox is not configured")
}
source := m.resolveSource(skillName)
// Get the skill base path
basePath, err := source.GetSkillBasePath(skillName)
if err != nil {
return nil, err
}
// Prepare execution config
logger.Info(ctx, "[Tool][ExecuteScript]:Prepare execution config")
sessionID, _ := types.SessionIDFromContext(ctx)
// Compute the artifact output directory. All skills share the same root
// directory (/workspace/output/) to enable collaboration and file sharing
// between different skill executions in the same session.
// Skill scripts read the directory via WEKNORA_SKILL_OUTPUT_DIR; the
// root (for cross-run discovery) is available via WEKNORA_SKILL_HISTORY_ROOT.
outputDir := m.SkillOutputDir(sessionID, skillName)
env := map[string]string{
artifactOutputEnvVar: outputDir,
artifactHistoryEnvVar: ArtifactOutputDir(),
}
applySessionPackagePath(env, skillName)
// SessionFileStore advertises the "sandbox provides per-session file
// storage" capability. When present we can safely expose the input
// staging directory and pre-materialise the output directory; when
// absent, directories are materialised during script execution.
fileStore := sessionFileStoreFromManager(m.sandboxMgr)
if fileStore != nil {
env[sessionInputEnvVar] = sandbox.SessionInputRoot
if sessionID != "" {
if err := fileStore.EnsureSessionDir(ctx, sessionID, outputDir); err != nil {
logger.Warnf(ctx, "[Tool][ExecuteScript] pre-create output dir %s failed: %v", outputDir, err)
}
}
}
// Per-caller values are resolved here rather than baked into the sandbox
// at creation: an IM thread can have several people sharing one sandbox,
// so each turn's values must belong to that turn's speaker and must not
// linger where the next person could read them with `env`.
//
// The resolver runs after the artifact keys above are seeded, so
// ApplyResolvedEnv's skip-existing rule protects WEKNORA_SKILL_OUTPUT_DIR
// and WEKNORA_SKILL_HISTORY_ROOT (always) plus WEKNORA_SESSION_INPUT_DIR
// (when a session file store exists). skillDirEnvVar is set later inside
// buildSkillExecuteConfig by an unconditional write. The write-time
// blacklist of skills.InjectedSandboxEnvVars (service.validateUserEnvName)
// is what covers a name that was not seeded on this path. Other WEKNORA_*
// names (API keys, base URLs) are ordinary credentials a skill may declare.
if m.envResolver != nil {
resolved, missing, err := m.envResolver.ResolveEnv(ctx, skillName)
if err != nil {
return nil, err
}
if len(missing) > 0 {
return nil, &MissingSkillEnvError{SkillName: skillName, Names: missing}
}
ApplyResolvedEnv(env, resolved)
}
config, err := buildSkillExecuteConfig(
source, skillName, scriptPath, basePath, args, stdin, env, sessionID,
)
if err != nil {
return nil, err
}
// Execute in sandbox
return m.sandboxMgr.Execute(ctx, config)
}
// buildSkillExecuteConfig turns one skill script into an execution request.
//
// Three cases:
// - A session /workspace file (from write_sandbox_file): run in place with
// the installed skill's interpreter. Preloaded skills cannot do this.
// - An installed image skill: run the relative path in place; no host upload.
// - A preloaded skill: upload from the host and run with the skill directory
// as WorkDir.
func buildSkillExecuteConfig(
source SkillSource,
skillName, scriptPath, basePath string,
args []string,
stdin string,
env map[string]string,
sessionID string,
) (*sandbox.ExecuteConfig, error) {
if workspace, ok := sandbox.RunnableWorkspaceScript(scriptPath); ok {
return workspaceSkillExecuteConfig(source, skillName, workspace, basePath, args, stdin, env, sessionID)
}
image, installed := source.(imageSkillSource)
if !installed {
// Load the script file to verify it exists and is a script
file, err := source.LoadSkillFile(skillName, scriptPath)
if err != nil {
return nil, fmt.Errorf("failed to load script: %w", err)
}
if !file.IsScript {
return nil, fmt.Errorf("file is not an executable script: %s", scriptPath)
}
return &sandbox.ExecuteConfig{
Script: file.Path,
Args: args,
WorkDir: basePath,
Stdin: stdin,
Env: env,
SessionID: sessionID,
}, nil
}
// The archive is deliberately not consulted: the image is what executes,
// and a skill whose archive failed to store is still installed and
// runnable. That leaves the extension as the only check available here,
// which is also the one the executor's interpreter choice depends on.
if !IsScript(scriptPath) {
return nil, fmt.Errorf("file is not an executable script: %s", scriptPath)
}
remoteScript, err := image.RemoteScriptPath(skillName, scriptPath)
if err != nil {
return nil, err
}
// The install-time verification pass exports the skill directory under
// this name, so the environment a script is checked in is the environment
// it is later called in.
env[skillDirEnvVar] = basePath
return &sandbox.ExecuteConfig{
RemoteScriptPath: remoteScript,
Args: args,
Stdin: stdin,
Env: env,
SessionID: sessionID,
}, nil
}
// workspaceSkillExecuteConfig runs a session-written /workspace file with the
// installed skill's interpreter. Preloaded (host-uploaded) skills have no
// in-sandbox venv to attach, so those calls are rejected with a shell_exec hint.
func workspaceSkillExecuteConfig(
source SkillSource,
skillName, workspacePath, basePath string,
args []string,
stdin string,
env map[string]string,
sessionID string,
) (*sandbox.ExecuteConfig, error) {
if !IsScript(workspacePath) {
return nil, fmt.Errorf("file is not an executable script: %s", workspacePath)
}
if _, installed := source.(imageSkillSource); !installed {
return nil, fmt.Errorf(
"script_path %q is a session workspace file; this skill is not installed in the sandbox image, so execute_skill_script cannot attach its environment. Run it with shell_exec, or pass a skill-relative path such as scripts/foo.py",
workspacePath,
)
}
skillDir, ok := sandbox.ValidatedImageSkillDir(basePath)
if !ok {
return nil, fmt.Errorf("cannot run workspace script %q for skill %q: no installed skill directory", workspacePath, skillName)
}
env[skillDirEnvVar] = skillDir
return &sandbox.ExecuteConfig{
RemoteScriptPath: workspacePath,
SkillDir: skillDir,
Args: args,
Stdin: stdin,
Env: env,
SessionID: sessionID,
}, nil
}
// sessionFileStoreFromManager returns the sandbox manager's effective
// session filesystem capability, or nil when the backend cannot expose one.
// Isolated in a helper so callers stay free of provider-specific branches.
func sessionFileStoreFromManager(mgr sandbox.Manager) sandbox.SessionFileStore {
provider, ok := mgr.(sandbox.SessionCapabilityProvider)
if !ok || provider == nil {
return nil
}
return provider.SessionFileStore()
}
// GetSkillInfo returns detailed information about a skill
func (m *Manager) GetSkillInfo(ctx context.Context, skillName string) (*SkillInfo, error) {
if !m.enabled {
return nil, fmt.Errorf("skills are not enabled")
}
if !m.isSkillAllowed(skillName) {
return nil, fmt.Errorf("skill not allowed: %s", skillName)
}
source := m.resolveSource(skillName)
skill, err := source.LoadSkillInstructions(skillName)
if err != nil {
return nil, err
}
files, err := source.ListSkillFiles(skillName)
if err != nil {
files = []string{} // Non-fatal error
}
return &SkillInfo{
Name: skill.Name,
Description: skill.Description,
BasePath: skill.BasePath,
Instructions: skill.Instructions,
Files: files,
}, nil
}
// SkillInfo provides detailed information about a skill
type SkillInfo struct {
Name string `json:"name"`
Description string `json:"description"`
BasePath string `json:"base_path"`
Instructions string `json:"instructions"`
Files []string `json:"files"`
}
// Reload refreshes the skill cache by rediscovering all skills
func (m *Manager) Reload(ctx context.Context) error {
if !m.enabled {
return nil
}
metadata, err := m.discoverAllSkills()
if err != nil {
return err
}
if len(m.allowedSkills) > 0 {
metadata = m.filterAllowedSkills(metadata)
}
m.mu.Lock()
m.metadataCache = metadata
m.mu.Unlock()
return nil
}
// Cleanup releases resources
func (m *Manager) Cleanup(ctx context.Context) error {
if m.sandboxMgr != nil {
return m.sandboxMgr.Cleanup(ctx)
}
return nil
}
+292
View File
@@ -0,0 +1,292 @@
// Package skills provides Agent Skills functionality following Claude's Progressive Disclosure pattern.
// Skills are modular capabilities that extend the agent's functionality through instruction files.
package skills
import (
"bufio"
"errors"
"fmt"
"path/filepath"
"regexp"
"strings"
"unicode/utf8"
)
// Skill validation constants following Claude's specification
const (
MaxNameLength = 64
MaxDescriptionLength = 1024
SkillFileName = "SKILL.md"
)
// Reserved words that cannot be used in skill names
var reservedWords = []string{"anthropic", "claude"}
// namePattern is the install identity: a single path segment of letters
// (any script), digits, hyphens, and underscores. Display titles such as
// "Word / DOCX" are not this; those are rewritten via slug / slugify
// before Validate. Slashes stay out so the name cannot walk SkillsImageRoot.
var namePattern = regexp.MustCompile(`^[\p{L}\p{N}_-]+$`)
var skillNameSepRE = regexp.MustCompile(`[^\p{L}\p{N}]+`)
// xmlTagPattern detects XML tags in content
var xmlTagPattern = regexp.MustCompile(`<[^>]+>`)
// Skill represents a loaded skill with its metadata and content
// It follows the Progressive Disclosure pattern:
// - Level 1 (Metadata): Name and Description are always loaded
// - Level 2 (Instructions): The main body of SKILL.md, loaded on demand
// - Level 3 (Resources): Additional files in the skill directory, loaded as needed
type Skill struct {
// Metadata (Level 1) - always loaded
Name string `yaml:"name"`
Description string `yaml:"description"`
// Slug is an optional filesystem-safe id. ClawHub / SkillHub often put a
// display title in name ("Word / DOCX") and the install id in slug
// ("word-docx"). When name is not a valid install identity, slug wins.
Slug string `yaml:"slug,omitempty"`
// Filesystem information
BasePath string // Absolute path to the skill directory
FilePath string // Absolute path to SKILL.md
// Instructions (Level 2) - loaded on demand
Instructions string // The main body of SKILL.md (after frontmatter)
Loaded bool // Whether Level 2 instructions have been loaded
// FrontmatterRepaired is true when the YAML between the --- markers had
// to be repaired (keys nested under a scalar, or an unquoted colon)
// before it would parse. The original SKILL.md is unchanged; callers
// that install the archive should tell the user so they can fix it.
FrontmatterRepaired bool
}
// SkillMetadata represents the minimal metadata for system prompt injection (Level 1)
// This is the lightweight representation used during skill discovery
type SkillMetadata struct {
Name string
Description string
BasePath string // Path to skill directory for later loading
}
// SkillFile represents an additional file within a skill directory (Level 3)
type SkillFile struct {
Name string // Filename (e.g., "FORMS.md", "scripts/validate.py")
Path string // Absolute path to the file
Content string // File content
IsScript bool // Whether this is an executable script
}
// Validate checks if the skill metadata is valid according to Claude's specification
func (s *Skill) Validate() error {
// Validate name
if s.Name == "" {
return errors.New("skill name is required")
}
if n := utf8.RuneCountInString(s.Name); n > MaxNameLength {
return fmt.Errorf("skill name is %d characters; maximum is %d", n, MaxNameLength)
}
if !namePattern.MatchString(s.Name) {
return errors.New("skill name must contain only letters, numbers, hyphens, and underscores")
}
for _, reserved := range reservedWords {
if strings.Contains(s.Name, reserved) {
return fmt.Errorf("skill name cannot contain reserved word: %s", reserved)
}
}
if xmlTagPattern.MatchString(s.Name) {
return errors.New("skill name cannot contain XML tags")
}
// Validate description
if s.Description == "" {
return errors.New("skill description is required")
}
if n := utf8.RuneCountInString(s.Description); n > MaxDescriptionLength {
return fmt.Errorf("skill description is %d characters; maximum is %d", n, MaxDescriptionLength)
}
if xmlTagPattern.MatchString(s.Description) {
return errors.New("skill description cannot contain XML tags")
}
return nil
}
// applyInstallName picks the directory / tool identity. Third-party SKILL.md
// files often put a display title in name ("Word / DOCX") and a kebab-case
// id in slug ("word-docx"). A title that is already a valid identity is
// left alone so Chinese names such as 律师助手 stay as-is.
func (s *Skill) applyInstallName() error {
if s == nil {
return errors.New("skill name is required")
}
if installableSkillName(s.Name) {
s.Name = strings.TrimSpace(s.Name)
return nil
}
if installableSkillName(s.Slug) {
s.Name = strings.TrimSpace(s.Slug)
return nil
}
if derived := slugifySkillName(s.Name); installableSkillName(derived) {
s.Name = derived
return nil
}
return errors.New("skill name must contain only letters, numbers, hyphens, and underscores (or set slug)")
}
func installableSkillName(name string) bool {
name = strings.TrimSpace(name)
return name != "" && namePattern.MatchString(name)
}
func slugifySkillName(name string) string {
s := strings.ToLower(strings.TrimSpace(name))
s = skillNameSepRE.ReplaceAllString(s, "-")
return strings.Trim(s, "-_")
}
// ToMetadata converts a Skill to its lightweight metadata representation
func (s *Skill) ToMetadata() *SkillMetadata {
return &SkillMetadata{
Name: s.Name,
Description: s.Description,
BasePath: s.BasePath,
}
}
// ParseSkillFile parses a SKILL.md file content and extracts metadata and body
// It handles YAML frontmatter enclosed in --- delimiters
func ParseSkillFile(content string) (*Skill, error) {
skill := &Skill{}
// Check for YAML frontmatter
if !strings.HasPrefix(strings.TrimSpace(content), "---") {
return nil, errors.New("SKILL.md must start with YAML frontmatter (---)")
}
// Find the end of frontmatter
scanner := bufio.NewScanner(strings.NewReader(content))
var frontmatterLines []string
var bodyLines []string
inFrontmatter := false
frontmatterEnded := false
for scanner.Scan() {
line := scanner.Text()
if !inFrontmatter && !frontmatterEnded && strings.TrimSpace(line) == "---" {
inFrontmatter = true
continue
}
if inFrontmatter && strings.TrimSpace(line) == "---" {
inFrontmatter = false
frontmatterEnded = true
continue
}
if inFrontmatter {
frontmatterLines = append(frontmatterLines, line)
} else if frontmatterEnded {
bodyLines = append(bodyLines, line)
}
}
if err := scanner.Err(); err != nil {
return nil, fmt.Errorf("error reading SKILL.md: %w", err)
}
if !frontmatterEnded {
return nil, errors.New("SKILL.md frontmatter is not properly closed with ---")
}
// Parse YAML frontmatter
frontmatter := strings.Join(frontmatterLines, "\n")
repaired, err := UnmarshalSkillFrontmatter(frontmatter, skill)
if err != nil {
return nil, fmt.Errorf("failed to parse YAML frontmatter: %w", err)
}
skill.FrontmatterRepaired = repaired
if err := skill.applyInstallName(); err != nil {
return nil, err
}
// Set body instructions
skill.Instructions = strings.TrimSpace(strings.Join(bodyLines, "\n"))
skill.Loaded = true
// Validate
if err := skill.Validate(); err != nil {
return nil, fmt.Errorf("skill validation failed: %w", err)
}
return skill, nil
}
// ParseSkillMetadata parses only the metadata from a SKILL.md file content
// This is a lightweight operation for skill discovery (Level 1 only)
func ParseSkillMetadata(content string) (*SkillMetadata, error) {
skill, err := ParseSkillFile(content)
if err != nil {
return nil, err
}
return skill.ToMetadata(), nil
}
// IsOnDemandInstallerPath reports whether path is a first-use dependency
// installer (scripts/install_deps.py and friends). Skills ship these to defer
// optional packages to chat time, which cannot work once the skill tree is
// snapshotted read-only. The installer agent's prompt ("install these extras
// now") and the runtime hint ("skip this installer") key off the same list.
func IsOnDemandInstallerPath(scriptPath string) bool {
base := strings.ToLower(filepath.Base(strings.TrimSpace(scriptPath)))
switch {
case strings.Contains(base, "install_dep"):
return true
case base == "setup_deps.py", base == "bootstrap_deps.py":
return true
default:
return false
}
}
// IsScript checks if a file path represents an executable script
func IsScript(path string) bool {
ext := strings.ToLower(filepath.Ext(path))
scriptExtensions := map[string]bool{
".py": true,
".sh": true,
".bash": true,
".js": true,
".mjs": true,
".cjs": true,
".ts": true,
".rb": true,
".pl": true,
".php": true,
}
return scriptExtensions[ext]
}
// GetScriptLanguage returns the language/interpreter for a script file
func GetScriptLanguage(path string) string {
ext := strings.ToLower(filepath.Ext(path))
languages := map[string]string{
".py": "python",
".sh": "bash",
".bash": "bash",
".js": "node",
".mjs": "node",
".cjs": "node",
".ts": "ts-node",
".rb": "ruby",
".pl": "perl",
".php": "php",
}
if lang, ok := languages[ext]; ok {
return lang
}
return "unknown"
}
+203
View File
@@ -0,0 +1,203 @@
package skills
import (
"fmt"
"reflect"
"regexp"
"strings"
"unicode"
"gopkg.in/yaml.v3"
)
// UnmarshalSkillFrontmatter decodes the YAML between SKILL.md's --- markers.
//
// Third-party skills (ClawHub / SkillHub) often indent `version` / `description`
// under `name:` as if it were a nested mapping, or leave a colon unquoted in
// a scalar. Strict YAML rejects both with "mapping values are not allowed in
// this context". When the first parse fails, those two conservative repairs
// are retried so those archives still install; valid frontmatter is unchanged.
//
// The decode always lands on a temporary value and is copied into dest only
// on success, so a failed candidate cannot leave dest half-written.
// repaired is true when a repair candidate was what succeeded.
func UnmarshalSkillFrontmatter(frontmatter string, dest any) (repaired bool, err error) {
firstErr := unmarshalFrontmatterCopy(frontmatter, dest)
if firstErr == nil {
return false, nil
}
for _, candidate := range frontmatterRepairCandidates(frontmatter) {
if candidate == frontmatter {
continue
}
if unmarshalFrontmatterCopy(candidate, dest) == nil {
return true, nil
}
}
return false, firstErr
}
func unmarshalFrontmatterCopy(src string, dest any) error {
rv := reflect.ValueOf(dest)
if rv.Kind() != reflect.Ptr || rv.IsNil() {
return fmt.Errorf("frontmatter dest must be a non-nil pointer")
}
tmp := reflect.New(rv.Elem().Type())
if err := yaml.Unmarshal([]byte(src), tmp.Interface()); err != nil {
return err
}
rv.Elem().Set(tmp.Elem())
return nil
}
func frontmatterRepairCandidates(frontmatter string) []string {
outdented := repairAccidentalNestedFrontmatter(frontmatter)
quoted := quoteColonInUnquotedScalars(frontmatter)
both := quoteColonInUnquotedScalars(outdented)
return []string{outdented, quoted, both}
}
// repairAccidentalNestedFrontmatter outdents keys that were nested under a
// plain scalar, e.g.
//
// name: 命理大师
// version: 1.2.6
// description: |
//
// A real nested mapping (`compatibility:` with no value on the same line) is
// left alone because it is valid YAML and does not need this pass.
func repairAccidentalNestedFrontmatter(src string) string {
lines := strings.Split(src, "\n")
out := make([]string, 0, len(lines))
for i := 0; i < len(lines); {
line := lines[i]
out = append(out, line)
i++
if !isPlainScalarMapping(strings.TrimSpace(line)) {
continue
}
indent := leadingWS(line)
if i >= len(lines) {
break
}
next := lines[i]
for i < len(lines) && strings.TrimSpace(lines[i]) == "" {
out = append(out, lines[i])
i++
if i < len(lines) {
next = lines[i]
}
}
if i >= len(lines) {
break
}
nextIndent := leadingWS(next)
if nextIndent <= indent || !looksLikeYAMLKey(strings.TrimSpace(next)) {
continue
}
extra := nextIndent - indent
for i < len(lines) {
cur := lines[i]
if strings.TrimSpace(cur) == "" {
out = append(out, cur)
i++
continue
}
curIndent := leadingWS(cur)
if curIndent < nextIndent {
break
}
out = append(out, stripLeadingWS(cur, extra))
i++
}
}
return strings.Join(out, "\n")
}
var unquotedColonScalarRE = regexp.MustCompile(
`^(\s*(?:name|description)\s*:\s*)([^"'|>{\[\s#].*:.+)$`,
)
// quoteColonInUnquotedScalars wraps name/description values that contain a
// colon so `description: Foo: bar` is not parsed as a nested mapping.
func quoteColonInUnquotedScalars(src string) string {
lines := strings.Split(src, "\n")
for i, line := range lines {
m := unquotedColonScalarRE.FindStringSubmatch(line)
if len(m) != 3 {
continue
}
value := strings.TrimSpace(m[2])
if strings.HasPrefix(value, `"`) || strings.HasPrefix(value, "'") {
continue
}
escaped := strings.ReplaceAll(value, `\`, `\\`)
escaped = strings.ReplaceAll(escaped, `"`, `\"`)
lines[i] = m[1] + `"` + escaped + `"`
}
return strings.Join(lines, "\n")
}
func isPlainScalarMapping(trimmed string) bool {
if trimmed == "" || strings.HasPrefix(trimmed, "#") {
return false
}
key, value, ok := strings.Cut(trimmed, ":")
if !ok || strings.TrimSpace(key) == "" {
return false
}
value = strings.TrimSpace(value)
if value == "" {
return false
}
if strings.HasPrefix(value, "#") {
return false
}
if value == "|" || value == ">" ||
strings.HasPrefix(value, "|") || strings.HasPrefix(value, ">") ||
strings.HasPrefix(value, "{") || strings.HasPrefix(value, "[") {
return false
}
return true
}
func looksLikeYAMLKey(trimmed string) bool {
if trimmed == "" || strings.HasPrefix(trimmed, "#") || strings.HasPrefix(trimmed, "-") {
return false
}
key, _, ok := strings.Cut(trimmed, ":")
if !ok {
return false
}
key = strings.TrimSpace(key)
if key == "" {
return false
}
if strings.HasPrefix(key, `"`) || strings.HasPrefix(key, "'") {
return true
}
for i, r := range key {
if i == 0 && !unicode.IsLetter(r) && r != '_' {
return false
}
if !unicode.IsLetter(r) && !unicode.IsDigit(r) && r != '_' && r != '-' && r != '.' {
return false
}
}
return true
}
func leadingWS(s string) int {
return len(s) - len(strings.TrimLeft(s, " \t"))
}
func stripLeadingWS(s string, n int) string {
if n <= 0 {
return s
}
i := 0
for i < len(s) && i < n && (s[i] == ' ' || s[i] == '\t') {
i++
}
return s[i:]
}
@@ -0,0 +1,131 @@
package skills
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/stretchr/testify/require"
)
func TestParseSkillFileRepairsKeysNestedUnderName(t *testing.T) {
content := `---
name: 命理大师
version: 1.2.6
description: |
全体系命理大师 — 八字四柱。
仅作文化参考。
metadata:
displayName: "命理大师"
version: 1.2.0
---
# Body
`
skill, err := ParseSkillFile(content)
require.NoError(t, err)
require.True(t, skill.FrontmatterRepaired)
require.Equal(t, "命理大师", skill.Name)
require.Contains(t, skill.Description, "全体系命理大师")
require.Contains(t, skill.Instructions, "# Body")
}
func TestParseSkillFileQuotesUnquotedColonInDescription(t *testing.T) {
content := `---
name: university-applications
description: Apply to universities: resume, essays, and interviews
---
Body
`
skill, err := ParseSkillFile(content)
require.NoError(t, err)
require.True(t, skill.FrontmatterRepaired)
require.Equal(t, "university-applications", skill.Name)
require.Equal(t, "Apply to universities: resume, essays, and interviews", skill.Description)
}
func TestRepairAccidentalNestedFrontmatterLeavesValidNestedMapping(t *testing.T) {
src := strings.TrimSpace(`
compatibility:
python: "3.11"
name: pdf-tools
description: Extract text
`)
require.Equal(t, src, repairAccidentalNestedFrontmatter(src))
var skill Skill
repaired, err := UnmarshalSkillFrontmatter(src, &skill)
require.NoError(t, err)
require.False(t, repaired)
require.Equal(t, "pdf-tools", skill.Name)
}
func TestUnmarshalSkillFrontmatterStillRejectsBrokenYAML(t *testing.T) {
var skill Skill
_, err := UnmarshalSkillFrontmatter(":\n - oops", &skill)
require.Error(t, err)
require.Empty(t, skill.Name)
}
func TestUnmarshalSkillFrontmatterDoesNotPolluteDestOnFailedRepair(t *testing.T) {
type dest struct {
Name string `yaml:"name"`
Description string `yaml:"description"`
Tags []string `yaml:"tags"`
}
// tags would decode, but the nested name/description plus broken
// metadata list is still invalid after the two repairs. dest must
// stay zero — not a mix of a partial first attempt and a later one.
fm := "tags:\n - a\n - b\nname: alpha\n version: 9.9.9\n description: |\n hello\nmetadata:\n openclaw:\n install:\n - kind: node\n package: iztro\n"
var d dest
repaired, err := UnmarshalSkillFrontmatter(fm, &d)
require.Error(t, err)
require.False(t, repaired)
require.Empty(t, d.Name)
require.Empty(t, d.Tags)
}
func TestParseSkillFileUsesSlugWhenNameIsADisplayTitle(t *testing.T) {
content := `---
name: Word / DOCX
slug: word-docx
description: Create and edit Microsoft Word documents.
---
Body
`
skill, err := ParseSkillFile(content)
require.NoError(t, err)
require.Equal(t, "word-docx", skill.Name)
}
func TestParseSkillFileSlugifiesDisplayTitleWithoutSlug(t *testing.T) {
content := `---
name: Word / DOCX
description: Create and edit Microsoft Word documents.
---
Body
`
skill, err := ParseSkillFile(content)
require.NoError(t, err)
require.Equal(t, "word-docx", skill.Name)
}
func TestParseSkillFileKeepsUnicodeName(t *testing.T) {
content := `---
name: 律师助手
description: 法律文书助手
---
Body
`
skill, err := ParseSkillFile(content)
require.NoError(t, err)
require.Equal(t, "律师助手", skill.Name)
}
func TestParseSkillFileRejectsSkillHubFrontmatterWithBrokenExtraYAML(t *testing.T) {
content, err := os.ReadFile(filepath.Join("testdata", "skillhub_nested_name.md"))
require.NoError(t, err)
_, err = ParseSkillFile(string(content))
require.Error(t, err, "nest repair is not enough when extra keys (metadata.openclaw.install) are still invalid YAML")
}
+446
View File
@@ -0,0 +1,446 @@
package skills
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
)
func TestParseSkillFile(t *testing.T) {
content := `---
name: test-skill
description: A test skill for unit testing purposes.
---
# Test Skill
This is the content of the test skill.
## Usage
Use this skill when testing.
`
skill, err := ParseSkillFile(content)
if err != nil {
t.Fatalf("Failed to parse skill file: %v", err)
}
if skill.Name != "test-skill" {
t.Errorf("Expected name 'test-skill', got '%s'", skill.Name)
}
if skill.Description != "A test skill for unit testing purposes." {
t.Errorf("Expected description 'A test skill for unit testing purposes.', got '%s'", skill.Description)
}
if skill.Instructions == "" {
t.Error("Expected instructions to be non-empty")
}
if !skill.Loaded {
t.Error("Expected Loaded to be true after parsing")
}
if skill.FrontmatterRepaired {
t.Error("valid frontmatter should not be marked repaired")
}
t.Logf("Parsed skill: name=%s, description=%s, instructions_len=%d",
skill.Name, skill.Description, len(skill.Instructions))
}
func TestSkillValidation(t *testing.T) {
tests := []struct {
name string
skillName string
description string
wantErr bool
errContains string
}{
{
name: "valid skill",
skillName: "my-skill",
description: "A valid skill",
wantErr: false,
},
{
name: "empty name",
skillName: "",
description: "A skill",
wantErr: true,
errContains: "name is required",
},
{
name: "invalid characters in name",
skillName: "My Skill",
description: "A skill",
wantErr: true,
errContains: "letters, numbers, hyphens, and underscores",
},
{
name: "underscore is allowed",
skillName: "word_docx",
description: "A skill",
wantErr: false,
},
{
name: "unicode letters are allowed",
skillName: "律师助手",
description: "A skill",
wantErr: false,
},
{
name: "reserved word in name",
skillName: "my-claude-skill",
description: "A skill",
wantErr: true,
errContains: "reserved word",
},
{
name: "empty description",
skillName: "my-skill",
description: "",
wantErr: true,
errContains: "description is required",
},
{
name: "name too long",
skillName: "this-is-a-very-long-skill-name-that-exceeds-the-maximum-allowed-length-of-64-characters",
description: "A skill",
wantErr: true,
errContains: "maximum is 64",
},
{
name: "description at 1024 CJK runes is allowed even though it is over 1024 bytes",
skillName: "my-skill",
description: strings.Repeat("描", 1024),
wantErr: false,
},
{
name: "description over 1024 CJK runes is rejected",
skillName: "my-skill",
description: strings.Repeat("描", 1025),
wantErr: true,
errContains: "maximum is 1024",
},
{
name: "description over 1024 ASCII runes is rejected",
skillName: "my-skill",
description: strings.Repeat("a", 1025),
wantErr: true,
errContains: "maximum is 1024",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
skill := &Skill{
Name: tt.skillName,
Description: tt.description,
}
err := skill.Validate()
if tt.wantErr {
if err == nil {
t.Errorf("Expected error containing '%s', got nil", tt.errContains)
} else if tt.errContains != "" && !containsString(err.Error(), tt.errContains) {
t.Errorf("Expected error containing '%s', got '%s'", tt.errContains, err.Error())
}
} else {
if err != nil {
t.Errorf("Unexpected error: %v", err)
}
}
})
}
}
func containsString(s, substr string) bool {
return len(s) >= len(substr) && (s == substr || len(s) > 0 && containsSubstring(s, substr))
}
func containsSubstring(s, substr string) bool {
for i := 0; i <= len(s)-len(substr); i++ {
if s[i:i+len(substr)] == substr {
return true
}
}
return false
}
func TestLoaderDiscoverSkills(t *testing.T) {
// Create a temporary skills directory
tmpDir, err := os.MkdirTemp("", "skills-test")
if err != nil {
t.Fatalf("Failed to create temp dir: %v", err)
}
defer os.RemoveAll(tmpDir)
// Create a test skill directory
skillDir := filepath.Join(tmpDir, "test-skill")
if err := os.MkdirAll(skillDir, 0755); err != nil {
t.Fatalf("Failed to create skill dir: %v", err)
}
// Write SKILL.md
skillContent := `---
name: test-skill
description: A test skill for loader testing.
---
# Test Skill
This is the test skill content.
`
if err := os.WriteFile(filepath.Join(skillDir, "SKILL.md"), []byte(skillContent), 0644); err != nil {
t.Fatalf("Failed to write SKILL.md: %v", err)
}
// Create loader and discover skills
loader := NewLoader([]string{tmpDir})
metadata, err := loader.DiscoverSkills()
if err != nil {
t.Fatalf("Failed to discover skills: %v", err)
}
if len(metadata) != 1 {
t.Fatalf("Expected 1 skill, got %d", len(metadata))
}
if metadata[0].Name != "test-skill" {
t.Errorf("Expected skill name 'test-skill', got '%s'", metadata[0].Name)
}
t.Logf("Discovered %d skills: %v", len(metadata), metadata[0].Name)
}
func TestLoaderLoadSkillInstructions(t *testing.T) {
// Create a temporary skills directory
tmpDir, err := os.MkdirTemp("", "skills-test")
if err != nil {
t.Fatalf("Failed to create temp dir: %v", err)
}
defer os.RemoveAll(tmpDir)
// Create a test skill directory
skillDir := filepath.Join(tmpDir, "test-skill")
if err := os.MkdirAll(skillDir, 0755); err != nil {
t.Fatalf("Failed to create skill dir: %v", err)
}
// Write SKILL.md
skillContent := `---
name: test-skill
description: A test skill for content loading.
---
# Test Skill
This is the main content.
## Section 1
More content here.
`
if err := os.WriteFile(filepath.Join(skillDir, "SKILL.md"), []byte(skillContent), 0644); err != nil {
t.Fatalf("Failed to write SKILL.md: %v", err)
}
// Create loader and load skill instructions
loader := NewLoader([]string{tmpDir})
skill, err := loader.LoadSkillInstructions("test-skill")
if err != nil {
t.Fatalf("Failed to load skill instructions: %v", err)
}
if skill.Name != "test-skill" {
t.Errorf("Expected skill name 'test-skill', got '%s'", skill.Name)
}
if skill.Instructions == "" {
t.Error("Expected instructions to be non-empty")
}
if !skill.Loaded {
t.Error("Expected Loaded to be true")
}
t.Logf("Loaded skill: name=%s, instructions_len=%d", skill.Name, len(skill.Instructions))
}
func TestLoaderLoadSkillFile(t *testing.T) {
// Create a temporary skills directory
tmpDir, err := os.MkdirTemp("", "skills-test")
if err != nil {
t.Fatalf("Failed to create temp dir: %v", err)
}
defer os.RemoveAll(tmpDir)
// Create a test skill directory with additional files
skillDir := filepath.Join(tmpDir, "test-skill")
scriptsDir := filepath.Join(skillDir, "scripts")
if err := os.MkdirAll(scriptsDir, 0755); err != nil {
t.Fatalf("Failed to create skill dir: %v", err)
}
// Write SKILL.md
skillContent := `---
name: test-skill
description: A test skill with additional files.
---
# Test Skill
See [GUIDE.md](GUIDE.md) for more info.
`
if err := os.WriteFile(filepath.Join(skillDir, "SKILL.md"), []byte(skillContent), 0644); err != nil {
t.Fatalf("Failed to write SKILL.md: %v", err)
}
// Write additional file
guideContent := "# Guide\n\nThis is the guide content."
if err := os.WriteFile(filepath.Join(skillDir, "GUIDE.md"), []byte(guideContent), 0644); err != nil {
t.Fatalf("Failed to write GUIDE.md: %v", err)
}
// Write a script
scriptContent := "#!/usr/bin/env python3\nprint('Hello from script')"
if err := os.WriteFile(filepath.Join(scriptsDir, "hello.py"), []byte(scriptContent), 0644); err != nil {
t.Fatalf("Failed to write script: %v", err)
}
// Create loader and discover skills first
loader := NewLoader([]string{tmpDir})
_, err = loader.DiscoverSkills()
if err != nil {
t.Fatalf("Failed to discover skills: %v", err)
}
// Load additional file
file, err := loader.LoadSkillFile("test-skill", "GUIDE.md")
if err != nil {
t.Fatalf("Failed to load skill file: %v", err)
}
if file.Content != guideContent {
t.Errorf("Expected guide content, got '%s'", file.Content)
}
if file.IsScript {
t.Error("GUIDE.md should not be marked as script")
}
// Load script file
scriptFile, err := loader.LoadSkillFile("test-skill", "scripts/hello.py")
if err != nil {
t.Fatalf("Failed to load script file: %v", err)
}
if !scriptFile.IsScript {
t.Error("hello.py should be marked as script")
}
t.Logf("Loaded files: GUIDE.md=%d bytes, hello.py=%d bytes (isScript=%v)",
len(file.Content), len(scriptFile.Content), scriptFile.IsScript)
}
func TestManagerIntegration(t *testing.T) {
// Create a temporary skills directory
tmpDir, err := os.MkdirTemp("", "skills-test")
if err != nil {
t.Fatalf("Failed to create temp dir: %v", err)
}
defer os.RemoveAll(tmpDir)
// Create a test skill directory
skillDir := filepath.Join(tmpDir, "test-skill")
if err := os.MkdirAll(skillDir, 0755); err != nil {
t.Fatalf("Failed to create skill dir: %v", err)
}
// Write SKILL.md
skillContent := `---
name: test-skill
description: A test skill for manager integration testing.
---
# Test Skill
Integration test content.
`
if err := os.WriteFile(filepath.Join(skillDir, "SKILL.md"), []byte(skillContent), 0644); err != nil {
t.Fatalf("Failed to write SKILL.md: %v", err)
}
// Create manager with config
config := &ManagerConfig{
SkillDirs: []string{tmpDir},
AllowedSkills: []string{}, // Allow all
Enabled: true,
}
manager := NewManager(config, nil) // No sandbox for this test
// Initialize
ctx := context.Background()
if err := manager.Initialize(ctx); err != nil {
t.Fatalf("Failed to initialize manager: %v", err)
}
// Get all metadata
metadata := manager.GetAllMetadata()
if len(metadata) != 1 {
t.Fatalf("Expected 1 skill, got %d", len(metadata))
}
// Load skill
skill, err := manager.LoadSkill(ctx, "test-skill")
if err != nil {
t.Fatalf("Failed to load skill: %v", err)
}
if skill.Name != "test-skill" {
t.Errorf("Expected skill name 'test-skill', got '%s'", skill.Name)
}
t.Logf("Manager integration test passed: %d skills discovered", len(metadata))
}
func TestIsScript(t *testing.T) {
tests := []struct {
path string
expected bool
}{
{"script.py", true},
{"script.sh", true},
{"script.bash", true},
{"script.js", true},
{"script.mjs", true},
{"script.cjs", true},
{"script.ts", true},
{"script.rb", true},
{"README.md", false},
{"data.json", false},
{"config.yaml", false},
}
for _, tt := range tests {
result := IsScript(tt.path)
if result != tt.expected {
t.Errorf("IsScript(%s) = %v, expected %v", tt.path, result, tt.expected)
}
}
}
func TestIsOnDemandInstallerPath(t *testing.T) {
for _, path := range []string{
"scripts/install_deps.py",
"scripts/install_dep.sh",
"setup_deps.py",
"bootstrap_deps.py",
} {
if !IsOnDemandInstallerPath(path) {
t.Errorf("IsOnDemandInstallerPath(%s) = false, want true", path)
}
}
if IsOnDemandInstallerPath("scripts/generate_ppt.py") {
t.Error("a real skill script must not look like an on-demand installer")
}
}
+32
View File
@@ -0,0 +1,32 @@
package skills
// SkillSource is where one set of skills comes from. Two implementations
// exist: the deployment's preloaded skill directories (*Loader) and the
// projection of the skills an administrator installed into a workspace sandbox
// config's snapshot image (*TenantSkillSource).
//
// The five methods are the Progressive Disclosure levels the agent asks for:
// metadata for the system prompt, the SKILL.md body, individual resource
// files, the file listing, and the directory a script runs from.
type SkillSource interface {
DiscoverSkills() ([]*SkillMetadata, error)
LoadSkillInstructions(name string) (*Skill, error)
LoadSkillFile(name, relativePath string) (*SkillFile, error)
ListSkillFiles(name string) ([]string, error)
GetSkillBasePath(name string) (string, error)
}
var _ SkillSource = (*Loader)(nil)
// imageSkillSource is a source whose skills already exist inside the sandbox
// image. It is what separates the two sources at execution time: a preloaded
// skill lives on the WeKnora host and has to be uploaded into the sandbox,
// while an installed skill is already there and is executed in place.
type imageSkillSource interface {
SkillSource
// RemoteScriptPath returns the absolute in-sandbox path of one script.
RemoteScriptPath(name, relativePath string) (string, error)
}
var _ imageSkillSource = (*TenantSkillSource)(nil)
+443
View File
@@ -0,0 +1,443 @@
package skills
import (
"archive/zip"
"bytes"
"fmt"
"io"
"os"
"path"
"sort"
"strings"
"sync"
"github.com/Tencent/WeKnora/internal/sandbox"
"github.com/Tencent/WeKnora/internal/types"
)
// Bundle limits bound decompression. They mirror the install-time limits: the
// archive read here is the same one that was accepted then, and a corrupted or
// hostile object in storage must not be able to exhaust this process.
const (
// maxBundleZipEntries bounds central-directory iteration on the raw
// archive, mirroring maxSkillBundleZipEntries. A GitHub zipball carries
// the whole repository, so it sits well above the count of files kept as
// the skill.
maxBundleZipEntries = 100_000
// maxBundleEntries caps the files kept as the skill, mirroring
// maxSkillBundleFiles. It is counted after directory entries and the
// zipball's extra trees are dropped, exactly as the install counts them:
// applying it to the raw entry count instead would reject archives the
// install accepted, taking read_skill down for a working install.
maxBundleEntries = 20_000
maxBundleEntryBytes = 32 << 20 // 32 MiB per entry
maxBundleBytes = 512 << 20 // 512 MiB across one archive (install cap)
)
// A source lives for a single agent run. read_skill is typically called
// several times against the same skill, so a handful of compressed zips
// removes the repeated download. 64 MiB is the keep-around budget; a zip
// larger than that can stay as the sole occupant so a large skill does not
// re-download on every file, without pinning several 100+ MiB archives.
const (
cachedBundleCount = 4
cachedBundleBytes = 64 << 20
)
// TenantSkillSource exposes the skills an administrator installed into one
// sandbox config's snapshot image.
//
// The three disclosure levels come from different places on purpose: name,
// description and the SKILL.md body are columns on the row, so the request
// path never waits on object storage to tell the model what a skill is, while
// individual resource files come from the uploaded archive. Execution uses
// neither - the files are already in the image.
type TenantSkillSource struct {
byName map[string]*types.TenantSkillEntity
// order preserves the repository's ordering so the system prompt is stable
// between turns.
order []string
// loadBundle fetches the skill zip. It may be nil when the deployment
// cannot serve bundles, in which case only the levels backed by the row
// are available. Callers that leave install.BundleRef empty should resolve
// the catalog object here.
loadBundle func(row *types.TenantSkillEntity) ([]byte, error)
mu sync.Mutex
// cache holds downloaded archives, most recently used first, keyed by
// bundle_sha256 (or the storage ref). The zip stays compressed: list and
// read_skill inflate one entry at a time so a 13k-file skill cannot pin
// hundreds of megabytes unpacked for the rest of the turn.
cache []cachedBundle
}
type cachedBundle struct {
key string
archive []byte
}
// NewTenantSkillSource builds a source over the rows of one sandbox config.
// Callers pass every row; the source itself decides which are usable.
func NewTenantSkillSource(
rows []*types.TenantSkillEntity, loadBundle func(row *types.TenantSkillEntity) ([]byte, error),
) *TenantSkillSource {
src := &TenantSkillSource{
byName: make(map[string]*types.TenantSkillEntity, len(rows)),
loadBundle: loadBundle,
}
for _, row := range rows {
if !usableSkillRow(row) {
continue
}
if _, exists := src.byName[row.Name]; exists {
continue
}
src.byName[row.Name] = row
src.order = append(src.order, row.Name)
}
return src
}
// usableSkillRow is the one place "the agent can actually run this" is decided
// for a single row. A row that is still installing, failed, or was disabled by
// an administrator is invisible: telling the model about a skill it cannot
// invoke costs it turns and gains nothing.
//
// The name guard is not defensive: every path this source hands out is
// SkillDirFor(row.Name), which joins the name under the skills root. A name
// that is not a single path segment yields a path outside the skill, or outside
// the root entirely, and those paths reach the model in discovery metadata and
// in SkillFile.Path even though execution would refuse them. Filtering the row
// out here is what keeps them from ever being spoken. sandbox.IsValidSkillName
// is the same rule SkillDirFor enforces, so this cannot disagree with it.
func usableSkillRow(row *types.TenantSkillEntity) bool {
return row != nil &&
row.Enabled &&
row.Status == types.SkillStatusReady &&
sandbox.IsValidSkillName(row.Name)
}
// DiscoverSkills returns Level 1 metadata for every usable skill.
func (s *TenantSkillSource) DiscoverSkills() ([]*SkillMetadata, error) {
metadata := make([]*SkillMetadata, 0, len(s.order))
for _, name := range s.order {
row := s.byName[name]
basePath, err := sandbox.SkillDirFor(row.Name)
if err != nil {
return nil, err
}
metadata = append(metadata, &SkillMetadata{
Name: row.Name,
Description: row.Description,
BasePath: basePath,
})
}
return metadata, nil
}
// LoadSkillInstructions returns Level 2 from the row.
func (s *TenantSkillSource) LoadSkillInstructions(name string) (*Skill, error) {
row, err := s.row(name)
if err != nil {
return nil, err
}
basePath, err := sandbox.SkillDirFor(row.Name)
if err != nil {
return nil, err
}
return &Skill{
Name: row.Name,
Description: row.Description,
BasePath: basePath,
FilePath: path.Join(basePath, SkillFileName),
Instructions: row.Instructions,
Loaded: true,
}, nil
}
// LoadSkillFile returns one Level 3 resource out of the uploaded archive.
//
// The archive is read rather than the image: reading a file out of the image
// would need a sandbox, and read_skill must work whether or not this turn has
// already booted one.
func (s *TenantSkillSource) LoadSkillFile(name, relativePath string) (*SkillFile, error) {
row, err := s.row(name)
if err != nil {
return nil, err
}
clean, err := safeSkillRelPath(relativePath)
if err != nil {
return nil, err
}
archive, err := s.bundleArchive(row)
if err != nil {
return nil, err
}
index, err := skillBundleFileIndex(archive)
if err != nil {
return nil, err
}
item, ok := index[clean]
if !ok {
return nil, fmt.Errorf("file not found in skill %s: %s", name, relativePath)
}
content, err := readLimitedSkillZipFile(item)
if err != nil {
return nil, err
}
basePath, err := sandbox.SkillDirFor(row.Name)
if err != nil {
return nil, err
}
return &SkillFile{
Name: relativePath,
// The path the model sees is the in-image one, because that is where
// it can act on the file.
Path: path.Join(basePath, clean),
Content: string(content),
IsScript: IsScript(clean),
}, nil
}
// ListSkillFiles lists the archive contents of one skill, sorted so repeated
// calls within a turn do not look like different answers.
func (s *TenantSkillSource) ListSkillFiles(name string) ([]string, error) {
row, err := s.row(name)
if err != nil {
return nil, err
}
archive, err := s.bundleArchive(row)
if err != nil {
return nil, err
}
index, err := skillBundleFileIndex(archive)
if err != nil {
return nil, err
}
names := make([]string, 0, len(index))
for rel := range index {
names = append(names, rel)
}
sort.Strings(names)
return names, nil
}
// GetSkillBasePath returns the skill's directory inside the image.
func (s *TenantSkillSource) GetSkillBasePath(name string) (string, error) {
row, err := s.row(name)
if err != nil {
return "", err
}
return sandbox.SkillDirFor(row.Name)
}
// RemoteScriptPath returns the absolute in-image path of one script. It is
// keyed on the skill name, which is also the directory the installer wrote.
//
// It deliberately does not consult the archive. The image is what executes,
// and a skill whose archive failed to store - the install logs a warning and
// carries on - is still installed and runnable.
func (s *TenantSkillSource) RemoteScriptPath(name, relativePath string) (string, error) {
row, err := s.row(name)
if err != nil {
return "", err
}
clean, err := safeSkillRelPath(relativePath)
if err != nil {
return "", err
}
basePath, err := sandbox.SkillDirFor(row.Name)
if err != nil {
return "", err
}
return path.Join(basePath, clean), nil
}
func (s *TenantSkillSource) row(name string) (*types.TenantSkillEntity, error) {
if row, ok := s.byName[name]; ok {
return row, nil
}
return nil, fmt.Errorf("skill not found: %s", name)
}
// safeSkillRelPath normalises a caller-supplied relative path and refuses
// anything that leaves the skill directory.
func safeSkillRelPath(relativePath string) (string, error) {
trimmed := strings.TrimSpace(relativePath)
if trimmed == "" {
return "", fmt.Errorf("skill file path is required")
}
if path.IsAbs(trimmed) {
return "", fmt.Errorf("invalid skill file path: %s", relativePath)
}
clean := path.Clean(trimmed)
if clean == "." || clean == ".." || strings.HasPrefix(clean, "../") {
return "", fmt.Errorf("invalid skill file path: %s", relativePath)
}
return clean, nil
}
// bundleArchive returns the compressed zip of one skill, downloading it at
// most once per cache lifetime.
func (s *TenantSkillSource) bundleArchive(
row *types.TenantSkillEntity,
) ([]byte, error) {
if s.loadBundle == nil {
return nil, fmt.Errorf("skill bundles are not available in this deployment")
}
key := strings.TrimSpace(row.BundleSHA256)
if key == "" {
key = strings.TrimSpace(row.BundleRef)
}
if key == "" {
key = strings.TrimSpace(row.CatalogID)
}
if key == "" {
key = strings.TrimSpace(row.ID)
}
if archive := s.cached(key); archive != nil {
return archive, nil
}
archive, err := s.loadBundle(row)
if err != nil {
return nil, fmt.Errorf("download bundle of skill %s: %w", row.Name, err)
}
if len(archive) == 0 {
return nil, fmt.Errorf(
"skill %s has no stored bundle; its files cannot be read", row.Name)
}
s.store(key, archive)
return archive, nil
}
func (s *TenantSkillSource) cached(key string) []byte {
s.mu.Lock()
defer s.mu.Unlock()
for i, entry := range s.cache {
if entry.key != key {
continue
}
s.cache = append(s.cache[:i], s.cache[i+1:]...)
s.cache = append([]cachedBundle{entry}, s.cache...)
return entry.archive
}
return nil
}
func (s *TenantSkillSource) store(key string, archive []byte) {
if key == "" || len(archive) == 0 || len(archive) > maxBundleBytes {
return
}
s.mu.Lock()
defer s.mu.Unlock()
for i, entry := range s.cache {
if entry.key != key {
continue
}
s.cache = append(s.cache[:i], s.cache[i+1:]...)
break
}
s.cache = append([]cachedBundle{{key: key, archive: archive}}, s.cache...)
for len(s.cache) > 1 &&
(len(s.cache) > cachedBundleCount || s.cachedBytes() > cachedBundleBytes) {
s.cache = s.cache[:len(s.cache)-1]
}
}
func (s *TenantSkillSource) cachedBytes() int {
total := 0
for _, entry := range s.cache {
total += len(entry.archive)
}
return total
}
// skillBundleFileIndex maps skill-root-relative paths to zip entries without
// inflating file bodies.
func skillBundleFileIndex(archive []byte) (map[string]*zip.File, error) {
reader, err := zip.NewReader(bytes.NewReader(archive), int64(len(archive)))
if err != nil {
return nil, fmt.Errorf("not a readable zip archive: %w", err)
}
if len(reader.File) > maxBundleZipEntries {
return nil, fmt.Errorf("archive holds more than %d entries", maxBundleZipEntries)
}
raw := make(map[string]*zip.File, len(reader.File))
for _, entry := range reader.File {
info := entry.FileInfo()
if info.IsDir() || info.Mode()&os.ModeSymlink != 0 {
continue
}
name := path.Clean(entry.Name)
if name == "." || path.IsAbs(name) ||
name == ".." || strings.HasPrefix(name, "../") {
return nil, fmt.Errorf("entry %q escapes the archive root", entry.Name)
}
raw[name] = entry
}
prefix, err := skillZipRootPrefix(raw)
if err != nil {
return nil, err
}
out := raw
if prefix != "" {
out = make(map[string]*zip.File, len(raw))
for name, entry := range raw {
if !strings.HasPrefix(name, prefix+"/") {
continue
}
out[strings.TrimPrefix(name, prefix+"/")] = entry
}
if _, ok := out[SkillFileName]; !ok {
return nil, fmt.Errorf("%s is missing from the archive", SkillFileName)
}
}
if len(out) > maxBundleEntries {
return nil, fmt.Errorf("archive holds more than %d files", maxBundleEntries)
}
return out, nil
}
func skillZipRootPrefix(raw map[string]*zip.File) (string, error) {
if _, ok := raw[SkillFileName]; ok {
return "", nil
}
prefix := ""
for name := range raw {
if path.Base(name) != SkillFileName {
continue
}
dir := path.Dir(name)
if dir == "." || strings.Contains(dir, "/") {
continue
}
if prefix != "" && prefix != dir {
return "", fmt.Errorf("archive holds more than one skill")
}
prefix = dir
}
if prefix == "" {
return "", fmt.Errorf("%s is missing from the archive", SkillFileName)
}
return prefix, nil
}
func readLimitedSkillZipFile(entry *zip.File) ([]byte, error) {
rc, err := entry.Open()
if err != nil {
return nil, fmt.Errorf("cannot read %q: %w", entry.Name, err)
}
content, err := io.ReadAll(io.LimitReader(rc, maxBundleEntryBytes+1))
_ = rc.Close()
if err != nil {
return nil, fmt.Errorf("cannot read %q: %w", entry.Name, err)
}
if len(content) > maxBundleEntryBytes {
return nil, fmt.Errorf("entry %q is too large", entry.Name)
}
return content, nil
}
+453
View File
@@ -0,0 +1,453 @@
package skills
import (
"archive/zip"
"bytes"
"context"
"errors"
"fmt"
"os"
"path/filepath"
"testing"
"github.com/Tencent/WeKnora/internal/sandbox"
"github.com/Tencent/WeKnora/internal/types"
"github.com/stretchr/testify/require"
)
func TestTenantSkillSourceOnlyExposesUsableSkills(t *testing.T) {
rows := []*types.TenantSkillEntity{
{ID: "sk-1", Name: "ready-enabled", Status: types.SkillStatusReady, Enabled: true},
{ID: "sk-2", Name: "ready-disabled", Status: types.SkillStatusReady, Enabled: false},
{ID: "sk-3", Name: "still-installing", Status: types.SkillStatusInstalling, Enabled: true},
{ID: "sk-4", Name: "failed", Status: types.SkillStatusFailed, Enabled: true},
}
src := NewTenantSkillSource(rows, nil)
metadata, err := src.DiscoverSkills()
require.NoError(t, err)
require.Len(t, metadata, 1,
"a skill the agent cannot actually run must never reach the system prompt")
require.Equal(t, "ready-enabled", metadata[0].Name)
}
// The name is joined under the skills root to build every path this source
// hands out, so it has to be a single path segment. Execution rejects an
// escaping path anyway, but the base path travels to the model in metadata
// and in SkillFile.Path, which is the same reasoning guardSkillDir applies
// on the install side.
func TestTenantSkillSourceRejectsANameThatIsNotOnePathSegment(t *testing.T) {
src := NewTenantSkillSource([]*types.TenantSkillEntity{
{ID: "sk-2", Name: "../escaping", Status: types.SkillStatusReady, Enabled: true},
{ID: "sk-3", Name: "nested/name", Status: types.SkillStatusReady, Enabled: true},
{ID: "sk-4", Name: "..", Status: types.SkillStatusReady, Enabled: true},
}, nil)
metadata, err := src.DiscoverSkills()
require.NoError(t, err)
require.Empty(t, metadata)
for _, name := range []string{"../escaping", "nested/name", ".."} {
_, err := src.GetSkillBasePath(name)
require.Error(t, err, "%s must be invisible, not merely unexecutable", name)
}
}
func TestTenantSkillSourceBasePathIsTheImageDir(t *testing.T) {
src := NewTenantSkillSource([]*types.TenantSkillEntity{
{ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true},
}, nil)
base, err := src.GetSkillBasePath("pdf")
require.NoError(t, err)
require.Equal(t, "/opt/weknora/tenant/skills/pdf", base,
"the path is the skill name: that is the directory the installer writes")
}
// A skill the source does not expose must be unreachable through every entry
// point, not just discovery: routing decisions elsewhere ask these methods.
func TestTenantSkillSourceHidesUnusableSkillsFromEveryLookup(t *testing.T) {
src := NewTenantSkillSource([]*types.TenantSkillEntity{
{ID: "sk-2", Name: "ready-disabled", Status: types.SkillStatusReady, Enabled: false},
}, func(*types.TenantSkillEntity) ([]byte, error) { return nil, errors.New("must not be called") })
_, err := src.GetSkillBasePath("ready-disabled")
require.Error(t, err)
_, err = src.LoadSkillInstructions("ready-disabled")
require.Error(t, err)
_, err = src.ListSkillFiles("ready-disabled")
require.Error(t, err)
_, err = src.LoadSkillFile("ready-disabled", "scripts/run.py")
require.Error(t, err)
_, err = src.RemoteScriptPath("ready-disabled", "scripts/run.py")
require.Error(t, err)
}
// Level 2 comes from the database projection: the request path must not depend
// on downloading an archive to answer "what does this skill say".
func TestTenantSkillSourceLoadsInstructionsFromTheRow(t *testing.T) {
src := NewTenantSkillSource([]*types.TenantSkillEntity{{
ID: "sk-1", Name: "pdf", Description: "PDF helpers",
Instructions: "Run scripts/extract.py.",
Status: types.SkillStatusReady, Enabled: true,
}}, func(*types.TenantSkillEntity) ([]byte, error) { return nil, errors.New("must not be called") })
skill, err := src.LoadSkillInstructions("pdf")
require.NoError(t, err)
require.Equal(t, "pdf", skill.Name)
require.Equal(t, "PDF helpers", skill.Description)
require.Equal(t, "Run scripts/extract.py.", skill.Instructions)
require.True(t, skill.Loaded)
require.Equal(t, "/opt/weknora/tenant/skills/pdf", skill.BasePath)
require.Equal(t, "/opt/weknora/tenant/skills/pdf/SKILL.md", skill.FilePath)
}
func TestTenantSkillSourceReadsLevel3FilesFromTheBundle(t *testing.T) {
archive := zipArchive(t, map[string]string{
// Wrapped in a top-level directory, which is how people upload.
"pdf-tools/SKILL.md": "---\nname: pdf\ndescription: d\n---\nbody\n",
"pdf-tools/reference/FORMS.md": "form notes",
"pdf-tools/scripts/extract.py": "print('hi')\n",
})
src := NewTenantSkillSource([]*types.TenantSkillEntity{{
ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true,
BundleRef: "local://sk-1.zip", BundleSHA256: "sha-1",
}}, func(row *types.TenantSkillEntity) ([]byte, error) {
require.Equal(t, "local://sk-1.zip", row.BundleRef)
return archive, nil
})
files, err := src.ListSkillFiles("pdf")
require.NoError(t, err)
require.Equal(t, []string{"SKILL.md", "reference/FORMS.md", "scripts/extract.py"}, files)
file, err := src.LoadSkillFile("pdf", "reference/FORMS.md")
require.NoError(t, err)
require.Equal(t, "form notes", file.Content)
require.Equal(t, "/opt/weknora/tenant/skills/pdf/reference/FORMS.md", file.Path,
"the path the model is shown must be the one it can execute or read in the sandbox")
require.False(t, file.IsScript)
script, err := src.LoadSkillFile("pdf", "scripts/extract.py")
require.NoError(t, err)
require.True(t, script.IsScript)
}
func TestTenantSkillSourceDownloadsEachBundleOnce(t *testing.T) {
archive := zipArchive(t, map[string]string{
"SKILL.md": "---\nname: pdf\ndescription: d\n---\nbody\n",
"scripts/extract.py": "print('hi')\n",
})
downloads := 0
src := NewTenantSkillSource([]*types.TenantSkillEntity{{
ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true,
BundleRef: "local://sk-1.zip", BundleSHA256: "sha-1",
}}, func(*types.TenantSkillEntity) ([]byte, error) {
downloads++
return archive, nil
})
_, err := src.ListSkillFiles("pdf")
require.NoError(t, err)
_, err = src.LoadSkillFile("pdf", "scripts/extract.py")
require.NoError(t, err)
require.Equal(t, 1, downloads,
"one read_skill call per file would re-download the whole archive every time")
}
func TestTenantSkillSourceRefusesPathsOutsideTheSkill(t *testing.T) {
src := NewTenantSkillSource([]*types.TenantSkillEntity{{
ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true,
BundleRef: "local://sk-1.zip",
}}, func(*types.TenantSkillEntity) ([]byte, error) {
return nil, errors.New("must not be called")
})
for _, rel := range []string{"../other/secret", "/etc/passwd", ""} {
_, err := src.LoadSkillFile("pdf", rel)
require.Error(t, err, "rel=%q", rel)
_, err = src.RemoteScriptPath("pdf", rel)
require.Error(t, err, "rel=%q", rel)
}
}
// A skill whose archive could not be stored is still installed and runnable in
// the image, so execution must not depend on the bundle being downloadable.
func TestTenantSkillSourceReportsAMissingBundleWithoutBlockingExecution(t *testing.T) {
src := NewTenantSkillSource([]*types.TenantSkillEntity{{
ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true,
}}, nil)
_, err := src.LoadSkillFile("pdf", "scripts/extract.py")
require.Error(t, err)
remote, err := src.RemoteScriptPath("pdf", "scripts/extract.py")
require.NoError(t, err)
require.Equal(t, "/opt/weknora/tenant/skills/pdf/scripts/extract.py", remote)
}
func TestManagerIgnoresPreloadedSkillsWhenTenantSourceIsAttached(t *testing.T) {
dir := preloadedSkillDir(t, "document-analyzer", "preloaded description")
mgr := NewManager(&ManagerConfig{SkillDirs: []string{dir}, Enabled: true}, nil)
mgr.WithTenantSource(NewTenantSkillSource([]*types.TenantSkillEntity{
{
ID: "sk-1", Name: "pdf", Description: "tenant description",
Instructions: "tenant body", Status: types.SkillStatusReady, Enabled: true,
},
{
ID: "sk-2", Name: "csv", Description: "tenant only",
Status: types.SkillStatusReady, Enabled: true,
},
}, nil))
require.NoError(t, mgr.Initialize(context.Background()))
byName := map[string]*SkillMetadata{}
for _, meta := range mgr.GetAllMetadata() {
byName[meta.Name] = meta
}
require.Len(t, byName, 2)
require.NotContains(t, byName, "document-analyzer",
"host preloaded skills are not in the sandbox image")
require.Equal(t, "tenant description", byName["pdf"].Description)
require.Equal(t, "tenant only", byName["csv"].Description)
skill, err := mgr.LoadSkill(context.Background(), "pdf")
require.NoError(t, err)
require.Equal(t, "tenant body", skill.Instructions)
_, err = mgr.LoadSkill(context.Background(), "document-analyzer")
require.Error(t, err, "a host-only skill must not be readable once the image is the source")
}
func TestManagerRunsATenantSkillFromTheImageWithoutUploading(t *testing.T) {
sandboxMgr := &recordingSandboxManager{}
mgr := NewManager(&ManagerConfig{SkillDirs: nil, Enabled: true}, sandboxMgr)
mgr.WithTenantSource(NewTenantSkillSource([]*types.TenantSkillEntity{{
ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true,
}}, nil))
require.NoError(t, mgr.Initialize(context.Background()))
_, err := mgr.ExecuteScript(
types.WithSessionID(context.Background(), "sess-1"),
"pdf", "scripts/extract.py", []string{"--flag"}, "",
)
require.NoError(t, err)
require.NotNil(t, sandboxMgr.config)
require.Equal(t, "/opt/weknora/tenant/skills/pdf/scripts/extract.py",
sandboxMgr.config.RemoteScriptPath)
require.Empty(t, sandboxMgr.config.Script,
"there is no host-side copy of an installed skill to upload")
require.Equal(t, []string{"--flag"}, sandboxMgr.config.Args)
require.Equal(t, "sess-1", sandboxMgr.config.SessionID)
require.Equal(t, sandbox.SkillsImageRoot+"/pdf",
sandboxMgr.config.Env["WEKNORA_SKILL_DIR"])
require.Equal(t, "/workspace/output", sandboxMgr.config.Env[artifactOutputEnvVar])
require.Equal(t, sandbox.SessionSkillPackageDir("pdf"),
sandboxMgr.config.Env["PYTHONPATH"])
require.Equal(t, sandbox.SessionSkillPackageDir("pdf"),
sandboxMgr.config.Env["NODE_PATH"])
}
func TestManagerRunsAWorkspaceScriptWithInstalledSkillInterpreter(t *testing.T) {
sandboxMgr := &recordingSandboxManager{}
mgr := NewManager(&ManagerConfig{SkillDirs: nil, Enabled: true}, sandboxMgr)
mgr.WithTenantSource(NewTenantSkillSource([]*types.TenantSkillEntity{{
ID: "sk-1", Name: "ppt-generator", Status: types.SkillStatusReady, Enabled: true,
}}, nil))
require.NoError(t, mgr.Initialize(context.Background()))
_, err := mgr.ExecuteScript(
types.WithSessionID(context.Background(), "sess-1"),
"ppt-generator", "/workspace/output/generate_rencui_ppt.py", []string{"--theme", "doraemon"}, "",
)
require.NoError(t, err)
require.NotNil(t, sandboxMgr.config)
require.Equal(t, "/workspace/output/generate_rencui_ppt.py", sandboxMgr.config.RemoteScriptPath)
require.Equal(t, sandbox.SkillsImageRoot+"/ppt-generator", sandboxMgr.config.SkillDir)
require.Empty(t, sandboxMgr.config.Script)
require.Equal(t, []string{"--theme", "doraemon"}, sandboxMgr.config.Args)
require.Equal(t, sandbox.SkillsImageRoot+"/ppt-generator",
sandboxMgr.config.Env["WEKNORA_SKILL_DIR"])
}
func TestManagerRejectsWorkspaceScriptForPreloadedSkill(t *testing.T) {
dir := preloadedSkillDir(t, "pdf", "preloaded description")
sandboxMgr := &recordingSandboxManager{}
mgr := NewManager(&ManagerConfig{SkillDirs: []string{dir}, Enabled: true}, sandboxMgr)
require.NoError(t, mgr.Initialize(context.Background()))
_, err := mgr.ExecuteScript(context.Background(), "pdf", "/workspace/output/custom.py", nil, "")
require.Error(t, err)
require.Contains(t, err.Error(), "shell_exec")
require.Zero(t, sandboxMgr.calls)
}
func TestManagerRejectsWorkspaceInputAsSkillScript(t *testing.T) {
sandboxMgr := &recordingSandboxManager{}
mgr := NewManager(&ManagerConfig{SkillDirs: nil, Enabled: true}, sandboxMgr)
mgr.WithTenantSource(NewTenantSkillSource([]*types.TenantSkillEntity{{
ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true,
}}, nil))
require.NoError(t, mgr.Initialize(context.Background()))
_, err := mgr.ExecuteScript(context.Background(), "pdf", "/workspace/input/upload.py", nil, "")
require.Error(t, err)
require.Zero(t, sandboxMgr.calls)
}
// The in-sandbox directory is what read_skill shows the model so it can name
// the skill's own interpreter instead of probing the system python3. A
// preloaded skill has no such directory: its base path is on the WeKnora host,
// which no command in the sandbox can reach.
func TestSandboxSkillDirOnlyAnswersForInstalledSkills(t *testing.T) {
installed := NewManager(&ManagerConfig{Enabled: true}, nil)
installed.WithTenantSource(NewTenantSkillSource([]*types.TenantSkillEntity{{
ID: "sk-1", Name: "pdf", Status: types.SkillStatusReady, Enabled: true,
}}, nil))
require.NoError(t, installed.Initialize(context.Background()))
dir, ok := installed.SandboxSkillDir("pdf")
require.True(t, ok)
require.Equal(t, sandbox.SkillsImageRoot+"/pdf", dir)
preloaded := NewManager(&ManagerConfig{
SkillDirs: []string{preloadedSkillDir(t, "pdf", "preloaded description")},
Enabled: true,
}, nil)
require.NoError(t, preloaded.Initialize(context.Background()))
_, ok = preloaded.SandboxSkillDir("pdf")
require.False(t, ok)
}
// Preloaded skills keep uploading from the host and keep running in their own
// directory; the tenant source must not change that path at all.
func TestManagerKeepsPreloadedSkillExecutionWhenNoTenantSource(t *testing.T) {
dir := preloadedSkillDir(t, "pdf", "preloaded description")
sandboxMgr := &recordingSandboxManager{}
mgr := NewManager(&ManagerConfig{SkillDirs: []string{dir}, Enabled: true}, sandboxMgr)
require.NoError(t, mgr.Initialize(context.Background()))
_, err := mgr.ExecuteScript(context.Background(), "pdf", "scripts/run.py", nil, "")
require.NoError(t, err)
require.NotNil(t, sandboxMgr.config)
require.Empty(t, sandboxMgr.config.RemoteScriptPath)
require.Equal(t, filepath.Join(dir, "pdf", "scripts", "run.py"), sandboxMgr.config.Script)
require.Equal(t, dir+"/pdf", sandboxMgr.config.WorkDir)
}
type recordingSandboxManager struct {
config *sandbox.ExecuteConfig
calls int
}
func (m *recordingSandboxManager) Execute(
_ context.Context, config *sandbox.ExecuteConfig,
) (*sandbox.ExecuteResult, error) {
m.calls++
m.config = config
return &sandbox.ExecuteResult{ExitCode: 0}, nil
}
func (m *recordingSandboxManager) Cleanup(context.Context) error { return nil }
func (m *recordingSandboxManager) GetSandbox() sandbox.Sandbox { return nil }
func (m *recordingSandboxManager) GetType() sandbox.SandboxType { return sandbox.SandboxTypeCube }
// preloadedSkillDir writes one deployment-preloaded skill to a temp directory
// and returns the search root it lives under.
func preloadedSkillDir(t *testing.T, name, description string) string {
t.Helper()
root := t.TempDir()
skillDir := filepath.Join(root, name)
require.NoError(t, os.MkdirAll(filepath.Join(skillDir, "scripts"), 0o755))
require.NoError(t, os.WriteFile(filepath.Join(skillDir, SkillFileName),
[]byte("---\nname: "+name+"\ndescription: "+description+"\n---\npreloaded body\n"), 0o644))
require.NoError(t, os.WriteFile(filepath.Join(skillDir, "scripts", "run.py"),
[]byte("print('preloaded')\n"), 0o644))
return root
}
func TestTenantSkillSourceCacheKeepsOneOversizeArchive(t *testing.T) {
src := NewTenantSkillSource(nil, nil)
src.store("small", bytes.Repeat([]byte("s"), 32))
src.store("big", bytes.Repeat([]byte("b"), cachedBundleBytes+1))
require.Equal(t, bytes.Repeat([]byte("b"), cachedBundleBytes+1), src.cached("big"))
require.Nil(t, src.cached("small"),
"a zip over the keep-around budget must not sit next to other entries")
}
// The agent reads the same archive the install accepted, so the two have to
// count it the same way. Counting raw zip entries against the skill-file cap
// rejects bundles the install took — directory entries alone can carry a real
// skill past 20k — and read_skill then fails on an install that works.
const bundleIndexSkillMD = "---\nname: pdf\ndescription: d\n---\nbody\n"
func TestSkillBundleFileIndexCountsFilesTheWayTheInstallDid(t *testing.T) {
files := map[string]string{"repo-main/" + SkillFileName: bundleIndexSkillMD}
for i := 0; i < maxBundleEntries-1; i++ {
files[fmt.Sprintf("repo-main/templates/asset-%d.txt", i)] = ""
}
// Directory entries push the raw count past the skill-file cap without
// adding a single file the skill is made of.
dirs := make([]string, 0, 2000)
for i := 0; i < 2000; i++ {
dirs = append(dirs, fmt.Sprintf("repo-main/templates/dir-%d/", i))
}
index, err := skillBundleFileIndex(zipArchiveWithDirs(t, files, dirs))
require.NoError(t, err)
require.Len(t, index, maxBundleEntries)
require.Contains(t, index, SkillFileName)
}
func TestSkillBundleFileIndexRejectsMoreSkillFilesThanTheCap(t *testing.T) {
files := map[string]string{"repo-main/" + SkillFileName: bundleIndexSkillMD}
for i := 0; i < maxBundleEntries; i++ {
files[fmt.Sprintf("repo-main/templates/asset-%d.txt", i)] = ""
}
_, err := skillBundleFileIndex(zipArchiveWithDirs(t, files, nil))
require.ErrorContains(t, err, "more than 20000 files")
}
func zipArchiveWithDirs(t *testing.T, files map[string]string, dirs []string) []byte {
t.Helper()
var buf bytes.Buffer
writer := zip.NewWriter(&buf)
for _, name := range dirs {
_, err := writer.Create(name)
require.NoError(t, err)
}
for name, content := range files {
entry, err := writer.Create(name)
require.NoError(t, err)
_, err = entry.Write([]byte(content))
require.NoError(t, err)
}
require.NoError(t, writer.Close())
return buf.Bytes()
}
func zipArchive(t *testing.T, files map[string]string) []byte {
t.Helper()
var buf bytes.Buffer
writer := zip.NewWriter(&buf)
for name, content := range files {
entry, err := writer.Create(name)
require.NoError(t, err)
_, err = entry.Write([]byte(content))
require.NoError(t, err)
}
require.NoError(t, writer.Close())
return buf.Bytes()
}
+86
View File
@@ -0,0 +1,86 @@
---
name: 命理大师
version: 1.2.6
description: |
全体系命理大师 — 八字四柱、紫微斗数、奇门遁甲、六爻、梅花易数、塔罗、星盘、
数字命理、九宫飞星风水、掌纹面相、起名命名、穿衣搭配、合婚择吉一站式解读。仅作文化参考,不替代医疗、法律、心理、财务、婚姻、命名等
专业建议;遇重大决策请咨询专业人士。
【数据与隐私提示】本 skill 会在本地文件系统保存用户主动提交的出生年月日时、地点、
姓名、可选的家庭成员信息、交互主题记录(用于追踪偏好,默认关闭),以及推送任务日志。
所有数据均不上传,仅在本地 data/ 目录留存;用户可随时通过 profile.js 查看、编辑或删除。
每日推送为 opt-in,默认关闭。
metadata:
displayName: "命理大师"
author:
- "腾讯高级研发-enoyao"
- "腾讯高级产品运营-rekyhe"
version: 1.2.0
keywords: 八字, 紫微斗数, 奇门遁甲, 六爻, 梅花易数, 塔罗, 星盘, 九宫飞星, 掌纹, 手相, 面相, 起名, 命名, 取名, 穿衣, 搭配, 颜色, 五行色, 开运色, 今日运势, 每日运程, 合婚, 择吉, 生命灵数, 风水, 算命, BaZi, ZiWei, QiMen, Tarot, feng shui, I Ching, numerology, daily horoscope, palmistry, physiognomy, naming, dressing
# 触发关键词收紧:1.1.8 移除「算命 / 占卜 / 命理 / 数字命理 / fortune telling / astrology」等过宽泛词,
# 避免在用户未明确要求占卜的对话中误激活本 skill;仅在明确出现具体体系名称或具体主题时才被触发。
openclaw:
emoji: "☯️"
skillKey: "university-applications"
runtime:
node: ">=18"
python3: true
install:
- kind: node
package: iztro
env: [] # 本 skill 不再读取任何环境变量;曾经的 OPENCLAW_KNOWLEDGE_DIR 已被移除以避免 file-system enumeration。
security:
network:
default: none
optional:
- feature: "liuyao HTML LLM divination (user-initiated, in-browser, OFF by default)"
allowed-endpoints:
- "https://api.openai.com"
- "https://api.anthropic.com"
- "https://api.deepseek.com"
custom-endpoint: "only after explicit in-UI consent dialog; HTTPS-only, no IP/localhost auto-trust"
data-sent: "only the hexagram and the user's typed question; no profile, no env vars, no file paths"
credential: "user-provided LLM API key, entered at runtime, stored in browser localStorage only"
- feature: "liuyao HTML Google Fonts (commented out by default)"
endpoint: "https://fonts.googleapis.com"
data-sent: "none beyond standard font request"
credential: none
credentials:
bundled: none
required: none
user-optional:
- "LLM API key for liuyao/index.html divination feature (scope it to a separate limited key, never reused)"
push-mechanism: openclaw-ipc
push-optin: true
push-default-state: disabled
data-retention:
location: "local filesystem under data/profiles/ and data/push-log.json"
remote-upload: none
user-controls:
- "view: node scripts/profile.js show <userId>"
- "list: node scripts/profile.js list"
- "edit: node scripts/profile.js save <userId> <field> <value>"
- "delete: node scripts/profile.js delete <userId>"
- "disable push: node scripts/push-toggle.js off <userId>"
notes: |
All bundled scripts perform local computation only — no fetch, axios,
https.request, curl, wget, or any outbound network calls from the Node/Python
side. Push delivery is handled entirely by the OpenClaw runtime via stdout/IPC
protocol, and is OPT-IN (disabled until the user runs push-toggle.js on).
The 'channels' field in user profiles (e.g. telegram) is a routing hint for
the OpenClaw runtime, not a direct API integration. This skill does not hold
or require any third-party API tokens (Telegram Bot Token, SMTP credentials,
webhook URLs, etc.). The local-only release helper script is excluded
from the published bundle via .clawhubignore and is not part of the
installed skill surface.
EXCEPTION — OPTIONAL LLM NETWORK USE: the browser-only file liuyao/index.html
exposes an optional "LLM divination" button. If and only if the user clicks
it and fills in their own API key + endpoint, the browser (not the skill
process) will POST the hexagram and question to that user-configured endpoint.
No key is bundled, hardcoded, or transmitted anywhere else. Users are advised
to supply a scoped/limited API key rather than a primary account key.
User profile data (birth details, optional family members, interaction log)
is stored only on the local filesystem and can be viewed, edited, or deleted
at any time via scripts/profile.js (see data-retention.user-controls above).
---
# body