mirror of
https://gh-proxy.org/https://github.com/klinsmaya/multica.git
synced 2026-11-08 01:16:45 +08:00
Danila Katalshov
ec6a7b1285
* fix(skills): skip binary supporting files in runtime-local skill sync collectLocalSkillFiles read every supporting file into SkillFileData.Content as a Go string. That string crosses the daemon/server hop as JSON, and encoding/json rewrites each invalid UTF-8 byte to U+FFFD, so writeSkillFiles recreated a file whose bytes differ from the original. A .docx, .xlsx or .pdf carried in a runtime-local skill bundle therefore arrived in the prepared task environment corrupt and would not open. The archive/URL importer already refuses this class of file for the mirror-image reason: those bytes cannot go into a PG TEXT column. Its extension blacklist moves to internal/skill as IsLikelyBinaryFilePath so both ingestion paths share one definition, and runtime-local collection now skips the same files with a log breadcrumb, matching the importer's behaviour. The skip does not depend on includeContent: discovery reports the bundle with content omitted and sync uploads it with content included, so a skip in only one pass would leave the two disagreeing about which files make up the bundle. Fixes #7143 * fix(skills): catch binary supporting files by content, not just extension collectLocalSkillFiles only skipped files whose extension matched a fixed blacklist, so a binary file with an unlisted or missing extension (.safetensors, .parquet, a stray no-extension blob) still went through SkillFileData.Content and came back corrupted on the other side. Read the file and require utf8.Valid on both the includeContent=false (discovery) and includeContent=true (sync) passes so they keep agreeing on the bundle; IsLikelyBinaryFilePath stays as the cheap extension-based first pass. Adds regression cases for an unlisted binary extension, a no-extension binary file, and a valid-UTF-8 text file that must still pass through untouched. * fix(skills): require NUL-free content, check extension before reading utf8.Valid alone is not a sufficient round-trip guarantee: a NUL byte is legal UTF-8, but the server-side import path strips every 0x00 via sanitizeNullBytes (server/internal/handler/skill_create.go), so a file that is valid UTF-8 but contains an embedded NUL still comes back different from what went in. UTF-16LE text made of ASCII characters is a realistic example. Require content to be both valid UTF-8 and NUL-free, with embedded_nul as its own skip reason distinct from invalid_utf8. Also reorders the extension check ahead of the read: IsLikelyBinaryFilePath is genuinely the cheap first pass now, so a known-binary extension never pays for the os.ReadFile it doesn't need. Adds regression cases for an unlisted extension and a no-extension file, each containing valid UTF-8 with an embedded NUL.
50 lines
1.4 KiB
Go
50 lines
1.4 KiB
Go
package skill
|
|
|
|
import "testing"
|
|
|
|
func TestIsLikelyBinaryFilePath(t *testing.T) {
|
|
tests := []struct {
|
|
path string
|
|
want bool
|
|
}{
|
|
// The Office formats runtime-local skill bundles actually carry.
|
|
{"references/report.docx", true},
|
|
{"references/data.xlsx", true},
|
|
{"deck.pptx", true},
|
|
{"legacy.doc", true},
|
|
{"spec.pdf", true},
|
|
// Other non-text payloads.
|
|
{"assets/logo.png", true},
|
|
{"fonts/Inter.woff2", true},
|
|
{"bundle.zip", true},
|
|
{"clip.mp4", true},
|
|
{"helper.so", true},
|
|
{"cache.sqlite3", true},
|
|
// Extension matching is case-insensitive: Windows skill dirs routinely
|
|
// carry upper-cased extensions.
|
|
{"references/REPORT.DOCX", true},
|
|
{"Assets/Logo.PNG", true},
|
|
// Text payloads must pass through — skipping these would silently drop
|
|
// the reference material a skill exists to provide.
|
|
{"SKILL.md", false},
|
|
{"references/notes.md", false},
|
|
{"scripts/run.sh", false},
|
|
{"config.json", false},
|
|
{"data.csv", false},
|
|
{"template.html", false},
|
|
{"main.py", false},
|
|
// No extension, and a dotfile whose name is not an extension.
|
|
{"Makefile", false},
|
|
{".gitignore", false},
|
|
{"", false},
|
|
// A directory-looking path with a binary extension is still binary.
|
|
{"nested/dir/archive.tar", true},
|
|
}
|
|
|
|
for _, tt := range tests {
|
|
if got := IsLikelyBinaryFilePath(tt.path); got != tt.want {
|
|
t.Errorf("IsLikelyBinaryFilePath(%q) = %v, want %v", tt.path, got, tt.want)
|
|
}
|
|
}
|
|
}
|