Files
multica/server/internal/service/plugin_storage.go
6a17c00144 MUL-6350: rebuild the plugin system (1/4) — cleanup and foundations (#7144)
* MUL-6350 refactor(plugins): reset the plugin system to Action / Hook / Resource foundations

The V1 plugin system shipped 14 tables, an append-only revision log per
relationship, a capability-snapshot compiler, a per-task execution manifest
pinned by an INSERT trigger on agent_task_queue, release signing, trust tiers,
and a bundled catalog — and never left its feature flag. It holds no production
data, so this removes it outright instead of migrating it forward.

What replaces it, per the design in MUL-6338:

- A plugin relates to Multica in exactly three ways. Action (plugin calls
  Multica), Hook (Multica calls plugin), Resource (a static contribution with no
  call at all). "Who triggers" and "what capability is called" are orthogonal:
  a hook is declared once and lists which triggers may invoke it.
- We never execute third-party code. A plugin runs only in a sandboxed iframe in
  the user's browser or on the author's own server.
- Distribution is "install by URL, admin reads the scopes". No signature, no
  trust tier, no publisher verification — the scope list IS the trust decision.

This change is the foundation, with no user-visible surface yet:

- Decouples remote MCP first. RemoteMCPConnection/RemoteMCPTool/DigestBytes move
  into pkg/remotemcp, where they belonged; the protocol layer and daemon broker
  survive intact and dormant, ready to be rewired for the agent trigger.
- Drops all 14 plugin_* tables, their triggers, the manifest-composition
  function, and agent_task_queue.plugin_execution_manifest_id. Adds three:
  plugin_installation (one row per workspace+plugin, holding the manifest
  snapshot the admin consented to), plugin_storage, plugin_secret.
- Defines the manifest schema in full, once — surfaces, hooks with their
  triggers and events, resources, and a closed scope list. Parsing stays strict
  (DisallowUnknownFields, no trailing JSON). Contribution kinds this build does
  not ship yet fail installation loudly rather than installing half-working.
- Two-step install: preview parses and returns the scopes without writing, then
  install requires consent matching the manifest exactly. Manifest fetches reuse
  the remote MCP SSRF guard; MULTICA_PLUGIN_DIR adds a local source for
  self-hosting and plugin authors.
- Storage quotas are enforced on write and never by eviction. Secrets live in a
  separate encrypted table with no read path that returns a value.

Co-authored-by: multica-agent <[email protected]>

* MUL-6350 fix(migrate): register the plugin index builds for invalid-index cleanup

Every migration that builds an index CONCURRENTLY must be registered in
concurrentIndexCleanups, or an interrupted build leaves a permanently INVALID
index that the `IF NOT EXISTS` retry records as success. TestEveryConcurrentUp\
BuildHasCleanup caught all three new plugin index migrations.

Co-authored-by: multica-agent <[email protected]>

* MUL-6350 fix(plugins): address review — byte quotas, version bound, secret pruning, script entry

From the review on #7144, plus one contract change the surface design forced.

- Storage quotas were accounted in characters and enforced in bytes. The usage
  query and the size_bytes it reports now use octet_length, and so do the
  plugin_storage column checks. A UTF-8 value could otherwise spend up to a
  quarter of the budget it appeared to.
- manifest version had no length bound while plugin_installation.version caps
  at 64, so a legal-but-long semver failed at INSERT as an opaque install
  error instead of a parse error naming the field.
- Upgrading pruned config values the new manifest dropped but left the matching
  plugin_secret rows. Ciphertext nothing can reach is exactly the residue
  pruneConfig exists to prevent; upgrade now prunes both in one transaction.
- SetConfig wrote secrets and config across two tables with no transaction, so
  a failed config write could leave a saved form half-applied.
- A duplicate first install mapped a unique-index violation to 502; it is 409.
- net: is now an exact host match. It was a suffix match in the hook transport
  check while the same scope becomes an exact-host CSP connect-src in the
  surface runtime — one scope string cannot mean two things across the two
  places the consent screen claims to describe.
- surfaces[].entry must be a .js/.mjs script, not an HTML document. The host
  generates the surface's document so it can attach the CSP derived from net:;
  a plugin-authored document would carry its own server's policy and net: would
  be a claim rather than a control.

Tests: the three storage limits get a canonical unit matrix plus DB-backed
coverage of the byte counting and the exclude-the-candidate-key replacement
semantics; upgrade-time secret pruning is pinned end to end (verified failing
without the fix); and the admin-only routes now have a real middleware-level
403 check, which every direct-handler test bypassed.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Lambda <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-18 19:15:56 +08:00

176 lines
6.5 KiB
Go

package service
import (
"context"
"errors"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgtype"
db "github.com/multica-ai/multica/server/pkg/db/generated"
)
// Plugin storage scopes. There are exactly two, and adding a third is a product
// decision, not a configuration knob.
const (
// PluginStorageWorkspace is team-shared state. scope_id is the workspace id.
PluginStorageWorkspace = "workspace"
// PluginStorageUser is per-member state, including that member's own
// credentials for the plugin's external service. scope_id is the user id.
PluginStorageUser = "user"
)
// Storage quotas. Enforced on write and never by eviction: a plugin that hits
// its ceiling gets an explicit error, because silently dropping the oldest key
// would corrupt state the plugin believes it stored.
const (
MaxPluginStorageKeyBytes = 1024
MaxPluginStorageValueBytes = 100 * 1024
MaxPluginStorageKeys = 1000
MaxPluginStorageTotalBytes = 5 * 1024 * 1024
)
// PluginStorageKey is one key in a listing. Values are deliberately absent:
// listing is for a plugin to discover what it wrote, not to bulk-read state.
type PluginStorageKey struct {
Key string `json:"key"`
SizeBytes int64 `json:"size_bytes"`
UpdatedAt string `json:"updated_at"`
}
// ResolveStorageScope maps a scope name to the row's scope_id.
func ResolveStorageScope(scopeType string, workspaceID, userID pgtype.UUID) (pgtype.UUID, error) {
switch scopeType {
case PluginStorageWorkspace:
return workspaceID, nil
case PluginStorageUser:
return userID, nil
default:
return pgtype.UUID{}, pluginErrf(PluginErrorInvalid, "storage scope must be %q or %q", PluginStorageWorkspace, PluginStorageUser)
}
}
func validateStorageKey(key string) error {
if key == "" {
return pluginErrf(PluginErrorInvalid, "storage key is required")
}
if len(key) > MaxPluginStorageKeyBytes {
return pluginErrf(PluginErrorQuota, "storage key exceeds %d bytes", MaxPluginStorageKeyBytes)
}
return nil
}
// GetStorageValue reads one value. This path can never reach plugin_secret:
// secrets live in a different table with no read query that returns ciphertext
// to a request handler.
func (s *PluginService) GetStorageValue(ctx context.Context, installationID pgtype.UUID, scopeType string, scopeID pgtype.UUID, key string) (string, error) {
if err := validateStorageKey(key); err != nil {
return "", err
}
row, err := s.Queries.GetPluginStorageValue(ctx, db.GetPluginStorageValueParams{
InstallationID: installationID,
ScopeType: scopeType,
ScopeID: scopeID,
Key: key,
})
if errors.Is(err, pgx.ErrNoRows) {
return "", pluginErrf(PluginErrorNotFound, "storage key not found")
}
if err != nil {
return "", &PluginError{Kind: PluginErrorUnavailable, Message: "read plugin storage", Err: err}
}
return row.Value, nil
}
func (s *PluginService) ListStorageKeys(ctx context.Context, installationID pgtype.UUID, scopeType string, scopeID pgtype.UUID) ([]PluginStorageKey, error) {
rows, err := s.Queries.ListPluginStorageKeys(ctx, db.ListPluginStorageKeysParams{
InstallationID: installationID,
ScopeType: scopeType,
ScopeID: scopeID,
})
if err != nil {
return nil, &PluginError{Kind: PluginErrorUnavailable, Message: "list plugin storage", Err: err}
}
keys := make([]PluginStorageKey, 0, len(rows))
for _, row := range rows {
keys = append(keys, PluginStorageKey{
Key: row.Key,
SizeBytes: row.SizeBytes,
UpdatedAt: row.UpdatedAt.Time.UTC().Format("2006-01-02T15:04:05Z07:00"),
})
}
return keys, nil
}
// EnforceStorageQuota decides whether one write fits, given the scope's usage
// with the candidate key excluded. Kept pure and separate from the write so the
// three limits have one canonical test, and so it is obvious that every bound
// is compared in bytes — the usage query reports octet_length for the same
// reason.
//
// Known bound: usage is read before the write without a lock, so concurrent
// writers to the same scope can overshoot by their own sizes. Serializing every
// plugin KV write to close that gap costs more than the overshoot is worth; the
// limits exist to stop runaway growth, not to be exact to the byte.
func EnforceStorageQuota(usage db.GetPluginStorageUsageRow, valueBytes int) error {
if valueBytes > MaxPluginStorageValueBytes {
return pluginErrf(PluginErrorQuota, "storage value exceeds %d bytes", MaxPluginStorageValueBytes)
}
if usage.KeyCount+1 > MaxPluginStorageKeys {
return pluginErrf(PluginErrorQuota, "storage scope already holds the maximum of %d keys", MaxPluginStorageKeys)
}
if usage.TotalBytes+int64(valueBytes) > MaxPluginStorageTotalBytes {
return pluginErrf(PluginErrorQuota, "storage scope exceeds its %d byte budget", MaxPluginStorageTotalBytes)
}
return nil
}
// SetStorageValue writes one value after checking every quota. The usage query
// excludes the key being written, so replacing a value is measured as a
// replacement and cannot fail a limit the existing row already occupies.
func (s *PluginService) SetStorageValue(ctx context.Context, installationID pgtype.UUID, scopeType string, scopeID pgtype.UUID, key, value string) error {
if err := validateStorageKey(key); err != nil {
return err
}
usage, err := s.Queries.GetPluginStorageUsage(ctx, db.GetPluginStorageUsageParams{
InstallationID: installationID,
ScopeType: scopeType,
ScopeID: scopeID,
Key: key,
})
if err != nil {
return &PluginError{Kind: PluginErrorUnavailable, Message: "read plugin storage usage", Err: err}
}
if err := EnforceStorageQuota(usage, len(value)); err != nil {
return err
}
if _, err := s.Queries.UpsertPluginStorageValue(ctx, db.UpsertPluginStorageValueParams{
InstallationID: installationID,
ScopeType: scopeType,
ScopeID: scopeID,
Key: key,
Value: value,
}); err != nil {
return &PluginError{Kind: PluginErrorUnavailable, Message: "write plugin storage", Err: err}
}
return nil
}
func (s *PluginService) DeleteStorageValue(ctx context.Context, installationID pgtype.UUID, scopeType string, scopeID pgtype.UUID, key string) error {
if err := validateStorageKey(key); err != nil {
return err
}
rows, err := s.Queries.DeletePluginStorageValue(ctx, db.DeletePluginStorageValueParams{
InstallationID: installationID,
ScopeType: scopeType,
ScopeID: scopeID,
Key: key,
})
if err != nil {
return &PluginError{Kind: PluginErrorUnavailable, Message: "delete plugin storage", Err: err}
}
if rows == 0 {
return pluginErrf(PluginErrorNotFound, "storage key not found")
}
return nil
}