Files
multica/server/internal/daemon/execenv/codex_user_skills.go
a1da8e536f MUL-6000 fix(daemon): link user Codex skills into codex-home instead of copying them (#6742)
* fix(daemon): link user Codex skills into codex-home instead of copying them

Every Codex task got a full recursive copy of ~/.codex/skills/ inside its
per-task CODEX_HOME. For a user with npm-backed skills that is ~100 MB per
task directory, re-copied on every task start — hydrateCodexSkills wipes and
re-seeds on both the Prepare and the Reuse path — and never reclaimed: the GC
artifact sweep only matches node_modules / .next / .turbo, and whole-directory
reclaim needs the issue to reach a terminal status first. Total cost scales
with the number of live task directories.

Link instead. The same per-task home already links its other two shared
resources, sessions and the plugin cache, and a skill directory is read-only
input to the CLI, so the write isolation a copy incidentally bought has no
requester. EvalSymlinks still runs first, so an installer-shipped symlink
(lark-cli into ~/.agents/skills/) resolves to the real directory rather than
producing a link to a link.

The three paths that could reach through the new link are unchanged in
behavior: os.RemoveAll deletes a link and not its target, writeSkillFiles
probes with os.Lstat so a workspace skill colliding with a linked user skill
lands in a `-multica` sibling, and the GC refuses to walk into links. Tests
pin all three, plus a windows-tagged test for the junction form of the link.

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): stop GC walks from descending into Windows directory junctions

The GC's artifact sweep and its size accounting both refuse to follow links
out of a task directory, but they tested for os.ModeSymlink alone. That misses
the link shape Windows actually produces: createDirLink falls back to a
directory junction (mklink /J) whenever os.Symlink is denied, which is the
default without Developer Mode, and since Go 1.23 os.Lstat reports a junction
as ModeDir|ModeIrregular with no ModeSymlink bit while its DirEntry still
answers IsDir() == true. filepath.WalkDir therefore descends straight into the
link target, and the sweep would delete a node_modules / .next / .turbo
directory that lives inside the user's real home.

Match both mode bits in all three walks. The task directory links user content
in three places today — the Codex session store, the plugin cache, and now the
user's skills — so this is reachable on any Windows host running Codex tasks.

Windows-tagged tests create the junction with mklink /J directly, so the
junction shape is covered even on a runner where symlinks are permitted, and
they run in the existing windows-execenv CI job.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Bohan-J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-11 13:51:50 +08:00

107 lines
4.2 KiB
Go

package execenv
import (
"fmt"
"log/slog"
"os"
"path/filepath"
"strings"
)
// seedUserCodexSkills links user-installed skill directories from the shared
// ~/.codex/skills/ into the per-task CODEX_HOME so the codex CLI discovers
// them natively. Codex is the only runtime whose HOME is redirected to a
// per-task directory (via the CODEX_HOME env var), so without this step the
// CLI never sees the user's `~/.codex/skills/` content.
//
// Links, never copies. A copy charges the whole skill tree to every task
// directory — 100 MB for a user with a handful of npm-backed skills — and
// hydrateCodexSkills re-does it on every task start, on the critical path,
// while no GC path ever reclaims it as long as the issue stays open. The two
// other shared resources in the same per-task home already link for the same
// reason: sessions (linkCodexSessionsToStore) and the plugin cache
// (exposeSharedCodexPluginCache). A skill directory is read-only input to the
// CLI, so the write isolation a copy incidentally bought has no requester.
//
// Workspace-assigned skills take precedence on name conflict: any user skill
// whose sanitized name matches a workspace skill's sanitized name is skipped
// here, and writeSkillFiles then writes the workspace version into a clean
// slot. writeSkillFiles never writes through a link it did not create:
// allocateCollisionFreeSkillDir probes with os.Lstat, so a link left here
// counts as occupied and the workspace skill lands in a `-multica` sibling.
//
// Per-skill failures are logged and skipped — a single broken user skill
// must not prevent the task from running. Returning an error is reserved for
// failures that would affect every skill: listing the shared skills
// directory, or creating the per-task skills directory.
func seedUserCodexSkills(codexHome string, workspaceSkills []SkillContextForEnv, logger *slog.Logger) error {
sharedSkillsDir := filepath.Join(resolveSharedCodexHome(), "skills")
info, err := os.Stat(sharedSkillsDir)
if err != nil {
if os.IsNotExist(err) {
return nil
}
return fmt.Errorf("stat shared skills dir: %w", err)
}
if !info.IsDir() {
return nil
}
reserved := make(map[string]struct{}, len(workspaceSkills))
for _, s := range workspaceSkills {
reserved[sanitizeSkillName(s.Name)] = struct{}{}
}
entries, err := os.ReadDir(sharedSkillsDir)
if err != nil {
return fmt.Errorf("read shared skills dir: %w", err)
}
targetSkillsDir := filepath.Join(codexHome, "skills")
for _, entry := range entries {
name := entry.Name()
if name == "" || strings.HasPrefix(name, ".") {
continue
}
if _, claimed := reserved[sanitizeSkillName(name)]; claimed {
logger.Info("execenv: codex user-skill yields to workspace skill", "name", name)
continue
}
src := filepath.Join(sharedSkillsDir, name)
// Installers like lark-cli ship each skill as a symlink into a
// shared ~/.agents/skills/<name>/ directory. Resolve it so the
// per-task link points at the real directory instead of at another
// link, and so the IsDir check below sees the actual target.
resolved, err := filepath.EvalSymlinks(src)
if err != nil {
logger.Warn("execenv: codex user-skill resolve failed", "name", name, "error", err)
continue
}
fi, err := os.Stat(resolved)
if err != nil || !fi.IsDir() {
continue
}
// hydrateCodexSkills wipes the skills dir before every seed, so the
// parent is normally absent here; createDirLink needs it to exist.
// Created lazily so a task with no eligible user skills still gets no
// skills directory at all.
if err := os.MkdirAll(targetSkillsDir, 0o755); err != nil {
return fmt.Errorf("create codex skills dir %s: %w", targetSkillsDir, err)
}
dst := filepath.Join(targetSkillsDir, name)
// Removes the link, never the link target: os.RemoveAll unlinks a
// symlink, and on Windows RemoveDirectory drops a junction while
// leaving the directory it points at intact.
if err := os.RemoveAll(dst); err != nil {
logger.Warn("execenv: codex user-skill clean dst failed", "name", name, "error", err)
continue
}
if err := createDirLink(resolved, dst); err != nil {
logger.Warn("execenv: codex user-skill link failed", "name", name, "error", err)
continue
}
}
return nil
}