Commit Graph
793 Commits
Author SHA1 Message Date
d276991c60 MUL-6743 refactor(issues): remove the assignment handoff note (#7626)
* refactor(issues): remove the assignment handoff note (MUL-6743)

Usage data puts the handoff note at 0.2% of assignments, so the box
occupied the confirm dialog's centre for the other 99.8% and bought
them nothing. Removing it takes the whole feature with it: the note
input and its runtime version gate, the `handoff_note` write field on
PATCH/batch issue updates, the task column write, and the daemon's
opening-prompt + issue_context.md handoff sections.

The confirm dialog itself stays — it still names who is about to run
and still offers "Don't start yet" (suppress_run).

The `*WithHandoff` enqueue methods only ever varied by that note, so
they become `EnqueueTaskForIssueByActor` / `EnqueueTaskForSquadLeaderByActor`
— the actor-carrying variants autopilot already called with an empty
note. `preview-trigger` drops `handoff_supported`, which nothing read
after the gate moved client-side.

`agent_task_queue.handoff_note` is deliberately left in place: it holds
historical rows and this is the expand half of an expand-contract pair.
Nothing reads or writes it now, so a follow-up migration can drop it
once no deployed binary selects it.

Co-authored-by: multica-agent <[email protected]>

* docs(issues): reshoot the assignment confirm dialog (MUL-6743)

The asset still showed the handoff note box the surrounding prose no
longer describes. Recaptured from the app at the same 600x450 frame;
the confirm button now carries the send-chord keycaps it has shipped
with since MUL-5694.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Lambda <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-09-03 16:13:33 +08:00
571e61128d MUL-7002: remove database reads from WebSocket heartbeats (#7976)
* fix(runtime): remove websocket heartbeat lookups

Co-authored-by: multica-agent <[email protected]>

* fix(runtime): preserve explicit deregistration

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Sol-Boy <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-09-03 16:12:34 +08:00
731af7ccd6 MUL-7002: push runtime deletion invalidation to active daemons (#7969)
* fix(runtime): invalidate deleted active runtimes

Co-authored-by: multica-agent <[email protected]>

* fix(runtime): separate invalidation delivery metrics

Co-authored-by: multica-agent <[email protected]>

* fix(runtime): complete invalidation metric separation

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Sol-Boy <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-09-03 14:50:14 +08:00
3df7c38ed1 fix(agent): normalise the provider prefix in the Hermes set_model skip-gate (MUL-6958) (#7908)
* fix(agent): normalise the provider prefix in the Hermes set_model skip-gate

The gate added in #5690 (MUL-5029) compares the configured model against the
model the session reports as current with a raw `==`:

    if opts.Model != "" && effectiveModel == sessionCurrentModel {

Those two values come from different id namespaces, so for an agent whose
`model` is stored bare the comparison can never be true. The gate is dead code
for those agents and `session/set_model` is replayed on every turn — which is
the exact situation #5690 was opened to prevent.

Why the two sides cannot match:

  - Hermes always reports a provider-encoded id. `_encode_model_choice` in its
    ACP adapter returns `f"{provider}:{model}"` whenever a provider resolves,
    and the provider falls back to `detect_provider()` or `"openrouter"`, so it
    is effectively never absent. Confirmed against a live `hermes acp` session:
    session/new populates `models.currentModelId` from that encoder, so a bare
    configured `claude-sonnet-4-5` on a `custom` provider is reported back as
    `custom:claude-sonnet-4-5`.

  - `agent.model` is persisted verbatim. `handler/agent.go` writes `req.Model`
    straight into the column on create and update; nothing normalises or
    validates a `provider:` prefix on that path.

  - The bare spelling is a documented, first-class input. `cmd_agent.go`'s own
    flag help is `--model  Model identifier (e.g. claude-sonnet-4-6,
    openai/gpt-4o)`. Only the ACP-catalog dropdown produces the prefixed form,
    because `acpModelEntry` uses the advertised id as `Model.ID`.

So dropdown-configured agents match and skip; every agent configured through
the CLI, the API, a seed, or a row predating the catalog does not, and never
will.

What the replay costs, per turn, in the Hermes runtime: `set_session_model`
replaces the session's agent with `_make_agent`, which re-runs `load_config` and
`resolve_runtime_provider`, joins background MCP discovery for up to
`mcp_discovery_timeout` (~1.5s), and constructs a fresh agent whose system
prompt is then re-derived from disk. It also re-runs `_resolve_model_selection`,
i.e. the provider auto-detection this gate exists to avoid — so the
`custom:deepseek-v4-pro` mis-route from #5690 is still live for precisely the
agents that hit it. Running Hermes' own resolver over the ids this daemon sends
reproduces it: `custom` + `deepseek-v4-pro` resolves to `openrouter`, and
`anthropic` + a bare Claude id resolves away from `anthropic`, both of which
make `_make_agent` drop base_url and api_mode.

The fix replaces the raw comparison with `acpModelIDsEquivalent`, which
normalises the optional prefix before comparing:

  - both sides carry an explicit provider → compare the whole id, so a genuine
    provider switch (openrouter:X → custom:X) still sends set_model;
  - exactly one side carries a provider → compare model names only; a bare
    configured id expresses no provider preference, so the session's own
    provider is authoritative;
  - either side empty → not equivalent, preserving the existing fall-through
    for runtimes that report no current model.

Only the first colon separates provider from model, so slash-bearing
OpenRouter-style names (`nous:moonshotai/kimi-k2.6`) split correctly, and the
comparison is case-insensitive because Hermes lowercases the provider when it
encodes the id.

No migration and no operator action. Rows already written in the prefixed form
keep working — both spellings now resolve to the same selection.

Deliberately not argued on cost. A cache-hit-rate difference first pointed here,
but that measurement was grouped by "does the agent have a configured model",
which on the measured deployment was confounded with the transport wire, so this
change does not claim it. It stands on the comparison being unsatisfiable and on
the guard it restores.

Tests: 12 `acpModelIDsEquivalent` cases, including the provider-switch and
different-model negatives, the case-folding and whitespace cases, and the
slash-bearing model id; plus 6 `splitACPModelID` cases covering the
leading-colon and multi-colon shapes.

* fix(agent): only skip set_model when an explicit provider is confirmed

Review caught a regression introduced by the previous commit: the third
equivalence rule was symmetric, but the risk is not.

    acpModelIDsEquivalent("openrouter:hermes-4", "hermes-4") // => true

Comparing model names whenever *either* side lacked a provider meant a caller
who named a provider explicitly got the switch swallowed, silently, and the
session kept whatever provider it had defaulted to. `origin/main` sends
set_model for that input, so the previous commit changed behaviour for it.

The premise that justified the symmetry — "Hermes always reports a
provider-encoded id" — is false, and this repository already contained the
counterexamples: `acp_effort_test.go` captures `currentModelId: "hermes-4"`
from `hermes acp` v0.20.0 and `currentModelId: "gpt-5.6-sol"` from jcode, both
unprefixed, and jcode routes through this same backend (hermes.go states it
covers both binaries). `_encode_model_choice` returning a prefix is what
happens when a provider resolves; it is not a property of every input.

The rule is now asymmetric:

  - bare configured id  -> equivalent on the model name alone. It expresses no
    provider preference, so the session's provider is authoritative. This is
    the case the gate exists for, and it still skips.
  - explicit configured provider -> equivalent only when the session reports
    that same provider. A bare current id cannot confirm the match, so we fall
    through and send set_model, as before this change.

Tests:

  - three `acpModelIDsEquivalent` table entries, including the missing
    `provider:model` vs bare-id row that let the regression through;
  - `TestHermesSkipsRedundantSetModelForBareConfiguredModel`, an end-to-end
    frame-recording test that asserts the gate actually fires for the bare
    spelling — the unit tests never checked that set_model stopped going out;
  - `TestHermesSendsSetModelWhenExplicitProviderAndBareCurrent`, which fails
    without the asymmetric rule (verified: the recorded frame is absent).

Also addressed from the same review:

  - dropped the stale cache-hit-rate paragraph from the `acpModelIDsEquivalent`
    comment. The PR description claimed it had been removed; it had not.
  - dropped the paragraph arguing the benefit via MCP refresh and tool-cache
    prefix invalidation. It described Hermes' Python internals this repository
    cannot verify or track, and it read as arguing against the sentence right
    after it.
  - softened "the comparison can never be true": it holds when the runtime
    reports a prefixed id, which is not always.
  - noted that `splitACPModelID` cannot distinguish a provider prefix from a
    colon inside a bare model name (`llama3:8b`), and that the degradation is
    safe in the only direction that matters — it falls back to sending
    set_model, never to a false skip.

* docs(agent): correct the bare-current-id evidence in the set_model gate

The justification for the asymmetric equivalence rule cited acp_effort_test.go
as capturing unprefixed model ids from both `hermes acp` and jcode. Only the
jcode half holds.

Verified against a live `hermes acp` v0.20.0 — the same version the fixture
names — driving `initialize` + `session/new`, the two calls this backend makes:
`models.currentModelId` came back provider-encoded in all three cases (no model
param, a bare model param, and a provider-encoded one), and all nine
availableModels ids were prefixed too. Sending a bare `gpt-5.6-terra` returned
`openai-codex:gpt-5.6-terra`, which is the defect this gate fixes and is now
first-hand rather than inferred.

Hermes Agent's encoder does have a bare branch — acp_adapter/server.py
`_encode_model_choice` returns the model alone when the provider is empty,
confirmed by calling it directly — but `_build_model_state` defaults the
provider (`state.agent.provider or detect_provider() or "openrouter"`) and
`normalize_provider("")` falls back to `openrouter`, so a normal install never
reaches it. The `hermes-4` in that fixture is better read as a simplification
made while writing a test about configOptions, not a verbatim capture.

The rule itself is unchanged and still correct: this backend serves Hermes
Agent and jcode, jcode really does report unprefixed ids, and refusing to
confirm an explicit provider against a bare current id costs nothing.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Garfield <[email protected]>
Co-authored-by: Bohan-J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-09-03 13:53:56 +08:00
879d0de916 refactor(agent tasks): drop the Active sibling runs prompt block (MUL-6984) (#7937)
Every issue turn carried a claim-time snapshot of the same agent's other
in-flight tasks, scoped only by (agent, workspace) and capped at five. The
scope had no relatedness filter, so the list answered "what else is this
agent running right now", not "what else touches the code you are about to
change".

Measured over one day of runs on this workspace: the block was injected 78
times across 30 runs, listing 96 distinct (current issue, sibling) pairs, and
not one pair shared a PR, a branch, or an issue family. `run-messages` was
invoked zero times off it. Meanwhile the day's one genuine overlap — a PR
review pushing to server/internal/metrics/pricing.go and a concurrent change
to the model catalog next to it — was never paired, because overlapping in
time is not the same as overlapping in code.

The duplicate-run defect this shipped with (#6947) is fixed by
shouldSuppressActiveSelfAssignment, which stops a second run on the same
(issue, agent) at the server. Once that holds, the block can only ever list a
DIFFERENT issue, so it was structurally unable to help with what it was added
for. #7922 then gave the coordination question a correct answer —
`issue runs --active` / `--siblings`, cross-agent, family-scoped, and read at
the moment it is asked — which is the same information as a pull instead of a
per-turn push.

Removed end to end rather than hidden behind a flag: the prompt block, the
claim payload field on both sides of the wire, and ListActiveSiblingIssueTasks.
The field was `omitempty` and the daemon already rendered nothing for an empty
list, so old and new server/daemon pairs stay compatible in both directions;
nothing outside server/ ever read active_sibling_runs.

Kept: the self-assignment suppression and its tests, `--no-start`,
`issue run-messages`, and `issue runs --active/--siblings`.

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-09-03 13:28:05 +08:00
603519d158 MUL-6977: test: remove implementation-only coverage (#7932)
* test: remove implementation-only coverage

Co-authored-by: multica-agent <[email protected]>

* test: remove remaining implementation-only coverage

Co-authored-by: multica-agent <[email protected]>

* test: restore compatibility and fragment navigation coverage

Co-authored-by: multica-agent <[email protected]>

* test: preserve behavioral cleanup contracts

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Sol-Boy <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-09-03 13:23:25 +08:00
765321f053 MUL-6951 fix(autopilot): carry the trigger creator's authorization on scheduled runs (#7918)
* fix(autopilot): carry the trigger owner's authorization on scheduled runs

A schedule/webhook autopilot run left originator_user_id NULL, so
canInvokeAgent saw no human and fail-closed against private agents. Arming
a trigger is the same authorizing act as clicking "run now" — deferred, not
absent — so withholding the originator did not remove that authority. It
only forced every capability the automation legitimately needed to arrive
as a separately-bounded borrow path: MUL-4857 for child creation, MUL-6691
for the run's own issues, #7902 for the accountable member's issues, with
sub-issue dispatch queued up as the fourth. Each carries a hand-written
scope that no operator can see in the UI.

attribution.TriggerOwner / RuleOwner now populate UserID (the originator)
instead of only the accountable side. finalizeAttribution mirrors it, so
the originator == accountable invariant that migrations 190/197 enforce
holds without a schema change or backfill. Both autopilot execution modes
resolve through triggerOwnerAttribution, so run_only and create_issue are
covered by the same two lines.

This grants exactly the trigger owner's own rights — canInvokeAgent is
untouched, only the principal handed to it changes — and originator_source
still distinguishes a scheduled fire from a manual one in the audit trail.

The borrowed-authority machinery in handler/agent_access.go is now dead for
newly enqueued runs but is deliberately left in place: tasks enqueued by
the previous backend are still running during a rolling deploy and carry
originator NULL, and they must keep the path they were admitted under.
Deleting it is a follow-up once this is deployed.

Composio connected-app credentials also key on originator_user_id, so
scheduled runs now receive the trigger owner's connected apps. Decoupling
that from authorization was considered and explicitly deferred.

Verification (real PostgreSQL 17):
- internal/attribution, internal/service, internal/handler: PASS
- the five attribution_stamp tests that pinned "originator stays NULL" were
  updated to assert the new principal, and fail against the old constructors
- pre-existing, unrelated failures in internal/daemon/execenv (4 Hermes
  store-path tests) reproduce identically on a clean tree

Co-authored-by: multica-agent <[email protected]>

* refactor(autopilot): delete the borrowed-authority machinery

With a scheduled run carrying its trigger owner as originator, every
autopilot-shaped authority path in agent_access.go is unreachable: the
invoke gate resolves a real human from the speaking task and judges it
exactly as it judges that human acting directly.

Removed:
  - autopilotTaskAssignAuthority + autopilotAssignAuthorityFromRequest (MUL-6691)
  - issueBoundToAutopilotTask, taskRunsAutopilotInWorkspace
  - autopilotDelegationAuthority and its FromRequest / FromComment resolvers (MUL-4857)
  - effectiveInvocationAuthorityFromRequest
  - assignAuthorityScope, assignScopeKind and the four scope constructors
  - commentTriggerComputeOptions.AutopilotDelegationAuthorityUserID + effectiveInvoker()

validateAssigneePair and triggerTasksForComment lose their now-meaningless
scope / delegation-authority parameters; every caller passes the run's
originator instead. This also retires the fourth borrow scope #7902 proposed.

CreateIssue still loads the parent issue when assigning, even though the gate
no longer needs the row: it is what turns a cross-workspace or missing parent
into 400 "parent issue not found" instead of a 403. Covered by a test.

Two behaviour notes, both accepted:

  - Authority now propagates down a delegation chain, because the parent run
    finally has an originator to copy. Under the old design each hop had to
    re-borrow. This matches what a manual "run now" already did.
  - The old borrow path required task.Status == "running"; the originator path
    does not, and never did for any non-autopilot run. Task tokens are revoked
    at the terminal transition (TaskService, MUL-2600), so the primary guard
    remains; this only drops a second layer that existed solely for autopilot
    runs.

Tests: the two suites that pinned the borrow bounds (autopilot-created-issue
binding, authored-by-this-task binding) described a mechanism that no longer
exists, so they collapse into one file asserting the rule that replaced them —
rights / no rights / no human at all, across create-top-level, create-child,
update and batch. The mention-authority suite keeps its shape; its fixture now
gives the run a human, as a live run has.

Verification (real PostgreSQL 17): internal/attribution, internal/service and
internal/handler all PASS. Pre-existing unrelated failures in cmd/multica,
internal/cli, internal/daemon and internal/daemon/execenv reproduce identically
on a clean tree.

Co-authored-by: multica-agent <[email protected]>

* fix(autopilot): resolve one principal per dispatch and expire it with the run

Addresses Elon's review of #7918. Three findings, all confirmed against the
code before fixing.

P1-1 — admission and the run's identity could be two different people.
Automatic dispatch admitted the assignee agent as the AUTOPILOT CREATOR
(canCreatorInvokeAgent) while the task took its originator from the trigger.
With A owning both the autopilot and a private agent, and B owning the
trigger, a dispatch was admitted as A and then executed as B — rights neither
holds alone. The old suite could not see it because it called dispatchRunOnly
directly, skipping admission.

  - autopilot_trigger gains an IMMUTABLE created_by (migration 449, best-effort
    backfill from published_by). published_by keeps its audit meaning and still
    transfers on a substantive edit; it is no longer an authorization input,
    because letting an editor move it silently re-authorizes the automation as
    that editor. Per Bohan's ruling the run always acts as the trigger CREATOR.
  - ResolveAutopilotTriggerPrincipal is now the single answer to "who does this
    run act as". Admission, the task stamp and every delegated run read it, so
    the fork is unrepresentable. It also binds the trigger to its autopilot
    (GetAutopilotTriggerForAutopilot — the old lookup was WHERE id = $1, which
    would let a foreign trigger select the principal) and re-checks workspace
    membership on every dispatch, so removing a member revokes their triggers.
  - canCreatorInvokeAgent is now unreachable and deleted.
  - RuleOwner reverts to accountable-only. It is a guess at who owns a rule;
    promoting it would hand a legacy trigger somebody's rights without that
    person arming anything. Such runs carry no originator and fail closed.

P1-2 — a terminal task could still spend its human's rights.
invokeOriginatorFromRequest now refuses completed / failed / cancelled tasks.
Token revocation at the terminal transition stays the primary guard, but it is
best-effort, explicitly non-fatal on failure, with a 24h expiry behind it. The
check sits at the authorization boundary rather than the auth boundary, so a
finishing run can still post its final comment but cannot start new work.

P1-3 — the PR and code comments described Composio backwards.
BuildTaskOverlay ignores the originator entirely and builds from the AGENT
OWNER's connections and allow-list (MUL-3963). Naming the originator hands out
no third-party credentials. Comments corrected; no decoupling was ever needed.

Operational note: a trigger with neither created_by nor published_by (predating
migration 186) resolves no principal, so its autopilot stops dispatching until
the trigger is re-saved. That is the intended fail-closed behaviour.

Tests: full dispatch through admission for both fork directions; principal
fails closed on legacy / cross-autopilot / removed-member; terminal task
refused on all three statuses; the edit-transfer tests now assert the principal
STAYS with the creator while published_by moves. Verified the fork test fails
when admission is reverted to canCreatorInvokeAgent.

Verification (real PostgreSQL 17, pristine database): internal/attribution,
internal/service, internal/handler and cmd/server all PASS. Migration 449
up/down round-trips. Pre-existing unrelated failures in cmd/multica,
internal/cli, internal/daemon and internal/daemon/execenv are unchanged.

Co-authored-by: multica-agent <[email protected]>

* fix(autopilot): bind the trigger principal to its workspace, correct stale docs

Second round of Elon's review on #7918.

1. The trigger lookup only enforced half of what its comment claimed.
GetAutopilotTriggerForAutopilot matched on (trigger id, autopilot id); the
workspace was used only for the creator's membership check afterwards. That
check proves the resolved human belongs to the workspace PASSED IN, not that
the autopilot does — a member of two workspaces satisfies it while the trigger
came from the other tenant. The query now joins autopilot and filters on
workspace_id, and the resolver passes it through. Regression test seeds two
workspaces sharing one member and asserts the foreign workspace resolves
nobody; verified it fails against the pre-fix query.

2. Several permission-boundary comments still described the pre-MUL-6951
model — that a run's authority and its accountable human transfer to whoever
last edited the trigger. That is now the opposite of the code, and these are
the comments a future reader would use to reason about the authorization
model, so they are corrected at all four sites: the trigger create site, the
SetAutopilotTriggerPublisher query, triggerOwnerAttribution, and the
dispatchRunOnly attribution block. published_by is described as what it now
is: a config-audit column with no bearing on a run.

3. The migration comment now states the backfill's imprecision plainly: for an
already-edited trigger it freezes the last recoverable EDITOR as the immutable
creator, not the historical creator, which the schema never recorded. It also
records that a NULL/NULL trigger has no recovery path by decision (Bohan:
leave them empty) — re-saving re-stamps published_by, not created_by — so its
runs stay refused with a recorded failure_reason.

Not done, per Bohan's ruling: no re-save takeover path for legacy triggers and
no migration-period alerting.

Verification (real PostgreSQL 17, pristine database): internal/attribution,
internal/service, internal/handler and cmd/server PASS; full suite shows only
the four pre-existing unrelated failures (cmd/multica, internal/cli,
internal/daemon, internal/daemon/execenv). go vet ./internal/... and
git diff --check clean.

Co-authored-by: multica-agent <[email protected]>

* docs(autopilot): carry the creator-principal semantics into every description

Third round of Elon's review on #7918. Two must-fixes, no behaviour change.

1. The final semantics had not reached every surface that describes them.

Migration 189 wrote COMMENT ON COLUMN text for autopilot_trigger.published_by_*
saying it decides the accountable human of the runs a trigger fires. That is
now false, and because sqlc generates column comments into models.go, the
stalest description of the authorization model sat in generated code every
reader passes through. 189 is released and untouched; migration 449 overrides
both comments and documents created_by_* for the first time, and sqlc is
regenerated. 449's down restores 189's original text verbatim so a rollback
does not leave MUL-6951 wording describing a model the database no longer runs.

The same stale semantics remained in prose at six code sites — the plain and
manual dispatch entry points, the scheduled-plan path, the create_issue enqueue
comment, the autopilot-level republish path, and the webhook trigger create
site. All said a run's authority or accountable human follows the last editor,
or that automation resolves rule_owner. These sit on the authorization path, so
they are corrected rather than left to drift. published_by is now described as
what it is: config audit, with no bearing on a run.

Also corrects the backfill comment's migration reference: trigger publishers
arrived in 189, not 186.

2. The test file this PR added open-coded INSERT ... RETURNING id with matching
t.Cleanup(DELETE ...), which CLAUDE.md prohibits. Rewritten onto
testutil.Fixture: a small principalFixture embeds it and every row — members,
runtimes, private agents, autopilots, triggers, the second workspace — is built
and cleaned up by dbfx. The only remaining DELETE is the mid-test member
removal that IS the behaviour under test. Re-verified after the rewrite that
both regressions still fail against the old code: reverting admission to the
autopilot-creator check reddens the fork test, and dropping the workspace join
reddens the cross-workspace test.

Verification (real PostgreSQL 17, pristine database): full suite shows only the
four pre-existing unrelated failures (cmd/multica, internal/cli,
internal/daemon, internal/daemon/execenv). Migration 449 up/down round-trips
and the down restores 189's comments. go vet ./internal/... and
git diff --check clean.

Co-authored-by: multica-agent <[email protected]>

* docs(autopilot): state the backfill widening plainly, drop raw-SQL test seeding

Fourth round of Elon's review on #7918. Both items are leftovers from the
previous round; no behaviour change.

1. Migration 449's backfill comment carried two factual errors.

It said published_by_* arrived in migration 186; it was 189. The previous round
intended to fix this but the string replacement did not match the wrapped line
and silently did nothing, so the wrong number shipped.

It also claimed the backfill "is no wider than the authority those runs carry
today", which is false: before MUL-6951 an automatic run had no originator at
all, only narrowly-scoped borrow paths, so promoting a backfilled editor to the
run's full originator IS a widening. The product decision to accept it stands;
the comment now says so directly as a compatibility tradeoff instead of denying
the widening. A migration comment that understates what it grants is worse than
no comment.

2. The new test file still reached raw SQL transitively.

It called seedAttributionFixture, which hand-writes INSERT and cleanup for its
user / workspace / member / runtime / agent — so the file's own claim that
"every row goes through testutil.Fixture" was untrue and the repository's
fixture rule was still unmet. It now bootstraps the base identity with
Fixture.User / Workspace / Member on an unbound fixture and adopts it, which
also drops the agent and issue that helper builds and these tests never use.
The comment explains why the helper is deliberately not reused.

Re-verified after the fixture change that both regressions still fail against
the old code: reverting admission to the autopilot-creator check reddens the
fork test, and dropping the workspace join reddens the cross-workspace test.

Verification (real PostgreSQL 17, pristine database): full suite shows only the
four pre-existing unrelated failures (cmd/multica, internal/cli,
internal/daemon, internal/daemon/execenv). go vet ./internal/..., gofmt and
git diff --check clean.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-09-02 18:29:11 +08:00
9fce92f427 MUL-6846: feat(cli): expose active and cross-issue agent runs on issue runs (#7922)
* feat(cli): expose active and cross-issue agent runs on `issue runs`

An agent could already read one issue's task runs, but only as the full
execution history — every past run, hydrated with attribution and token
usage — with no way to ask the one question coordination actually needs:
who is working right now. And nothing at all could answer it across a
parent's sub-issues, which is the case #7768 describes: several agents
claim sibling children in parallel and each starts overlapping work
without knowing the others exist.

The claim-time `## Active sibling runs` block does not cover this. It is
scoped by agent_id — your OWN other in-flight tasks — so a different
agent running on a sibling issue is invisible, and it is a snapshot taken
before the turn started.

Two query params on the existing task-runs endpoint, and the flags that
drive them:

  multica issue runs <id> --active      # in-flight runs on this issue
  multica issue runs <id> --siblings    # ...and across the sub-issue family

`--siblings` roots at the issue's parent, or at the issue itself when it
has none. One rule covers both directions: a child sees its siblings, a
parent sees its children, a standalone issue degenerates to its own
active runs — so a caller does not have to know an issue's shape before
choosing a flag. Rows carry the issue they belong to, ordered
running-first and bounded, because a parent that fanned out hundreds of
children must not turn an advisory read into an unbounded response.

Both reads are advisory. Nothing here reserves an issue or serialises
anything; they tell a run whom to coordinate with, and coordination
still happens in comments.

With neither param the response is byte-identical to before, which the
issue-detail execution log and the CLI's short-task-ID resolver both
depend on.

Co-authored-by: multica-agent <[email protected]>

* fix(cli): address review on active/sibling run reads

Four fixes from review, all on the same theme: a coordination read has to
be cheap and has to be honest about what it did not tell you.

- `--active` no longer hydrates usage. ListIssueTaskUsage returns a row
  per (task, provider, model) for every task the issue ever ran, so the
  active path was still paying the full-history cost the filter exists to
  remove — for a column that is near-empty on runs that have not
  finished. The execution log keeps it.

- An invalid `active` value is a 400 instead of a silent fallback.
  `active=tru` used to answer "who is here now?" with the whole history
  under a 200, and a pile of finished runs reads as the live ones. Same
  reason the unknown-scope check already existed, same shape as the
  comment-list boolean params.

- The 50-row family cap now announces itself. The query asks for one row
  past the cap and sets X-Active-Runs-Truncated when it comes back; the
  CLI warns on stderr. Without it, "the answer was cut off" and "no run
  on that sibling" are the same response, and an agent reads the second.
  The cap is also documented in the flag help, the skill and
  CLI_AND_DAEMON.md, along with the ordering it depends on.

- The skill's new agent-facing contract is pinned in
  builtin_skills_test.go: that the claim-time block is same-agent only,
  the `--siblings` command itself, the cap disclosure, and the
  advisory/no-lock boundary — the last a negative anchor, because an
  agent that reads these as a lock skips the coordination they exist to
  prompt.

Also reuses service.IssueIdentifier instead of re-deriving the format.

Co-authored-by: multica-agent <[email protected]>

* fix(server): expose the truncation header to CORS, correct the cap rationale

Both from re-review, and both about the cap telling the truth.

- HeaderActiveRunsTruncated was missing from corsExposedHeaders. No
  browser client requests scope=family today, so nothing is broken right
  now — but a custom response header that is not exposed is unreadable
  from JS, which is the exact silent-truncation failure the header was
  added to remove. The registry and its guard test exist because this
  drifted once already (MUL-5492); completing both is two lines.

- The comment justifying 50 was wrong. It claimed real family
  concurrency is bounded far below the cap so 50 sits above any
  legitimate answer. It does not: agentconfig.MaxMaxConcurrentTasks lets
  ONE agent run 50 concurrent tasks, this feature exists for the case
  where several agents work a family at once, and the returned set
  includes queued / dispatched / waiting rows that no execution slot
  bounds at all — a parent that fans out 200 children can have 200
  enqueued. So 50 is a response-size budget and nothing more, truncation
  is an ordinary outcome rather than a pathological one, and that is
  precisely why it has to be reported. Comment and PR body now say that.

Co-authored-by: multica-agent <[email protected]>

* refactor(cli): give the family read its own compact row, cap at 20

The family read was reusing AgentTaskResponse — the execution log's row.
Measured against real data that is ~1645 bytes per run, of which the
caller reads almost nothing: attribution (625 bytes, and its own extra
query), result, work_dir, relative_work_dir, trigger_summary,
coalesced comment ids. An agent asking "is anyone working next to me?"
was spending ~20k tokens of its own context on the answer.

ActiveRunSummary carries what the question needs and nothing else: task,
issue, agent, status, and timestamps. 367 bytes per row. It is a separate
type on purpose — not AgentTaskResponse, and not ActiveSiblingRunData
(the claim payload's row, which has no agent field because those runs all
belong to the claiming agent by construction). Reusing either would mean
an optional field only one caller ever sets, which is the shape this
commit removes from AgentTaskResponse.

The query now names its columns instead of embedding the task row, so
Postgres stops detoasting the JSONB blobs too, and the handler drops
attribution hydration along with the field.

Cap goes 50 -> 20 (maintainer call): a handful of runs is the normal
case, so the budget is set to the normal case. Truncation was already
reported and still is, which is what makes a small cap safe.

Worst-case response: 80 KB / ~20k tokens -> 7 KB / ~1.8k tokens.

The CLI's family table drops COMPLETED and ERROR, which are empty on
every row of a read that only returns work still in flight.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-09-02 16:21:05 +08:00
a075e58b80 MUL-6961 feat(agent): discover Claude Code's model catalog from the CLI (#7917)
* feat(agent): discover Claude Code's model catalog from the CLI

Claude was the last first-party runtime answering "what models do you
have?" from a hand-maintained list. A static catalog cannot know what the
CLI in front of it can run, so every new Anthropic model opened a window
where Multica offered a model that 400s:

    API Error: 400 Claude Code 2.1.246 does not support this model;
    version 2.1.251 or newer is required.

Claude Code does have a discovery hook, just not on the command line: the
stream-json control protocol answers a `list_models` control request with
the rows its own /model picker renders. Ask the installed binary and the
window closes at the source — a model it cannot run is either absent or
flagged disabled, computed against the real CLI version and the logged-in
account.

Verified against 2.1.223/228/246/258. The three older builds return Fable
5.1 as a disabled row carrying "Update to 2.1.255+ to use Fable 5.1";
2.1.258 returns it as a normal pick. Those rows are the reason no
per-model minimum-version table is needed here: upstream already computes
one, including the remedy copy, which we forward verbatim and grey out.

No version gate guards the request. An unsupported subtype is answered,
not ignored — every build tested replies "Unsupported control request
subtype" in ~2s and exits 0 — so an old CLI costs one cheap round trip and
falls back. Gating would mean hand-maintaining exactly the kind of number
this change removes.

Notes:

- The static catalog stays, now as the fallback and as the server-side
  accept list. It is flagged Catalog.Fallback so a discovery failure is
  not pinned in the server's day-long cache (MUL-5549).
- Discovery replaces rather than merges, as with every other dynamic
  provider. The picker narrows to the CLI's own list; agents already
  pinned to an older model keep working, and manual entry still reaches
  them.
- Effort levels now come from each row instead of scraping `claude
  --help` and narrowing through the hand-kept claudeModelEffortAllow.
- Catalog ids keep the context-window tag, because `claude-opus-5[1m]` is
  what the CLI would really run for that row. That exposed a latent bug
  in two capability lookups, which normalized the query side but not the
  catalog side; both now normalize both, or a model would fail to find
  its own effort picker.

MUL-6961

Co-authored-by: multica-agent <[email protected]>

* fix(agent): keep unrunnable Claude models out of the selectable catalog

Review found the first cut carried "this CLI cannot run it" as a flag on
the model itself. That was wrong in a way a flag cannot fix.

Only the create-agent dropdown learned the flag. The inspector picker and
the agent builder read the same list and kept offering the row, so
editing an existing agent still reached the 400 this change exists to
prevent. And the part no amount of teaching fixes: an already installed
desktop client has never heard of the field, so it renders the row as an
ordinary model and persists a model id the CLI rejects — exactly what the
repo's mixed-version API rule forbids.

Unavailable models now travel in their own list, end to end: daemon
report, store, catalog cache, and the client result. `models` contains
only selectable entries. The inspector and the builder needed no changes
to become correct — they read `models`, and the rows are no longer in it
— and an old client is correct for the same reason. Both pickers render
the advisory rows through one shared component that takes no click
handler, so nothing can turn them back into controls.

Two more review findings:

- The default row was dropped when no sibling row named the same model,
  which an org-restricted install can produce. That is not one missing
  picker entry: ValidateThinkingLevelWith resolves an empty model — the
  common "follow the CLI default" configuration — through the entry
  flagged Default, and fails closed without one, silently discarding the
  user's effort on every such task. It is now materialised, keeping the
  row's own capabilities. The test that claimed to cover this asserted
  the opposite of its own comment; rewritten, plus a case proving an
  unrunnable default is never materialised.

- "One cheap round trip" for a CLI without list_models was one round trip
  PER TASK: every Claude task carrying a thinking_level loads the catalog
  before it starts, and cachedDiscovery refuses to memoise a fallback.
  The structured unsupported-subtype answer is now remembered per
  (command, CLI version) — a property of the binary, not a bad moment —
  so an old CLI is probed once. Transient failures stay uncached and
  retryable, and the version in the key makes an upgrade take effect at
  once rather than after a TTL. Detecting it costs ~0.01s against the
  ~1.7s it saves, and the fallback path already paid for it.

The discovery timeout keeps its 20s ceiling, now with the measurements
that justify it: 1.57-2.19s over five runs against 2.1.258, 1.64-1.79s
against 2.1.246. The ceiling is for a CLI that never answers, not a slow
one.

Verified against real binaries: 2.1.246 offers four selectable models and
one unavailable row carrying upstream's own upgrade hint, 2.1.258 offers
Fable 5.1 normally, a missing binary falls back to the flagged static
catalog.

MUL-6961

Co-authored-by: multica-agent <[email protected]>

* docs(agent): record the downgrade case as an accepted boundary

Dynamic discovery governs selection, and only selection. An agent already
configured with a model whose CLI is later downgraded still starts and
still gets the upstream 400 — discovery cannot reach a value already
persisted.

That is deliberate, not an oversight, so it is written down next to the
code that invites the wrong fix. Clearing the configured model turns a
legible error into a silent config change the user did not make;
substituting a runnable one is worse, because it succeeds using a model
nobody chose. The downgrade is rare and the upstream error is accurate,
so failing loudly at task start is a fine way to report it.

MUL-6961

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-09-02 15:48:38 +08:00
a2d819ea40 MUL-6962 fix(daemon): give environment preparation failures their own failure reason (#7915)
* fix(daemon): give environment preparation failures their own failure reason

A failed execenv.Prepare / Reuse is a filesystem error on the daemon's own
host, but handleTask routed it through taskfailure.Classify — a classifier
written to read agent and provider output — so a full disk, a denied
permission or a directory held open by another process all landed in
agent_error.*: the catchall today, provider_server_error on the report that
opened the issue and sent hours of diagnosis at an LLM vendor.

The daemon now tags those errors structurally (environmentSetupError) and
reports the new platform-side reason environment_prepare_failed. The four
existing sentinels still win: task identity, the prepare budget, skill
downloads and the OpenClaw CLI deadline each name a specific cause inside the
same phase.

Installed daemons update on their own cadence, and they report a non-empty
reason that FailTask deliberately leaves alone, so NormalizeDaemonReason
upgrades any agent_error.* value whose error text opens with the prepare /
reuse wrapper. That witness proves the run died before an agent existed, so
every label in that namespace is wrong by construction.

Not retryable and resume-safe: a full disk reproduces identically on the next
attempt, and no session was touched.

Closes #7913

Co-authored-by: multica-agent <[email protected]>

* fix(mobile): parse failure_reason as an open string, and match copy to real scope

Review follow-up on #7915.

AgentTaskSchema pinned failure_reason to the six pre-MUL-1949 coarse values
with `.catch("")`, so every refined reason was erased to undefined before any
surface read it. run-row's badge map was widened in MUL-5370 and has been
unreachable ever since: agent_error.*, skill_bundle_unavailable and the new
environment_prepare_failed all rendered a bare "Failed". The field is an open
string on the wire (packages/core/types/agent.ts) and in the chat message
schema beside it; the task schema now matches, and the badge map moved to
lib/run-failure-badge.ts so mobile's node-only vitest lane can cover the
lookup a component test cannot reach.

execenv.Prepare is not only filesystem work — it also writes and validates the
per-provider local config (Codex home, Hermes overlay, Cursor MCP, and an
OpenClaw config it fails closed on), so a malformed local config lands on this
reason too. The Go docs, the chat copy in four locales and the docs table said
"disk, permission, directory in use", which would send the reader past half
the real causes — the same misdirection this reason exists to end. They now
point at the raw error first and name the config case, with regression tests
for a non-filesystem preparation failure on both the daemon and the legacy
wire path.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-09-02 14:45:41 +08:00
Rusty Raven 2e424e06a1 MUL-5154: fix(agent): surface Kimi provider API errors (#5785)
Surface Kimi provider failures that ACP reports as successful empty turns, preserve terminal error details across retries and EOF, and connect poisoned-history failures to the existing guarded fresh-session recovery path.
2026-09-02 14:45:39 +08:00
yarrumretep 9d09902dfe MUL-6943 feat(agent): add Claude Fable 5.1 model and pricing (#7892)
Adds `claude-fable-5-1` to the static Claude Code model catalog and to the
backend and frontend pricing tables, so the model is selectable with its
effort picker enabled and its usage prices at the published rates
($10 input / $50 output / $12.50 5m cache write / $0.25 cache read — 5.1
prices cache hits at 0.025x input rather than the standard 0.1x).

Maintainer follow-ups on top of the contribution tighten backend alias
matching: both Fable rules now end at a shared `claudeVersionEnd`
terminator anchored to the end of the id, and the tag-stripping retry in
`PriceForModelAlias` peels exactly one context tag. Together these stop
Fable 5 and 5.1 from inheriting each other's tier, keep an unknown variant
(`claude-fable-5-2`, `-10`, `-1-preview`, `[1m][2m]`) unmapped instead of
silently borrowing a rate, and keep the backend and the dashboard resolving
the same suffix forms. Regression coverage added for the dated, `-latest`,
context-tagged and doubly-tagged spellings.

Sonnet 4.6 remains the sole default. Known follow-up: Claude Code rejects
this model below 2.1.251 while the global Claude runtime floor is 2.0.0, so
members on an older daemon see it in the picker and fail at run time; the
fix is a per-model minimum-runtime-version signal, tracked separately.
2026-09-02 13:38:15 +08:00
3c23c491fa MUL-6924: feat(billing): notify workspaces about autopilot quota (#7681)
* feat(billing): notify workspaces about autopilot quota

Co-authored-by: multica-agent <[email protected]>

* fix(billing): isolate and throttle quota notices

Co-authored-by: multica-agent <[email protected]>

* fix(billing): tighten quota notice delivery

Co-authored-by: multica-agent <[email protected]>

* fix(billing): reuse quota admission snapshot

Co-authored-by: multica-agent <[email protected]>

* fix(autopilot): make quota notices actionable

Co-authored-by: multica-agent <[email protected]>

* fix(billing): align quota recovery notifications

Co-authored-by: multica-agent <[email protected]>

* fix(billing): align quota notice recipients

Co-authored-by: multica-agent <[email protected]>

* fix(billing): finish quota notice review

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: multica-agent <[email protected]>
Co-authored-by: Sol-Boy <[email protected]>
2026-09-02 12:59:26 +08:00
f01eb5a06c MUL-6928 perf(issues): index scalar property contains behind a bigram prefilter (#7878)
* MUL-6928 perf(issues): index scalar property contains behind a bigram prefilter

Scalar `contains` property filtering runs `(properties ->> def) ILIKE
'%needle%'`, which no index can serve, so it degrades to a workspace-wide
sequential scan. At 1M issues that costs ~8x the throughput of an equality
filter, and the pgxpool connections it holds queue every other tenant's cheap
query behind it.

Put a redundant, indexable prefilter in front of the per-key check:
`LOWER(properties::text) LIKE LOWER('%needle%')`, served by a new pg_bigm GIN
index on the same expression. The per-key ILIKE still decides the result; the
prefilter only narrows the candidate set. On a 500k-row pg_trgm proxy a
selective needle goes from a 183ms Parallel Seq Scan to a 1.4ms Bitmap Index
Scan.

The prefilter is skipped, falling back to today's behaviour, for needles it
cannot screen soundly or usefully: jsonb escapes `"`, `\` and control
characters on serialization, so those never appear literally in
properties::text, and a single character carries no bigram to narrow with.

Migration 446 cannot use the DO ... EXCEPTION guard migrations 032 / 036 use
for pg_bigm — a concurrent build cannot run inside one — so the runner gates it
on the operator class instead: databases without pg_bigm record the version
with its SQL skipped and keep the unaccelerated (still correct) filter.
cmd/migrate/README.md documents building it out of band afterwards.

Co-authored-by: multica-agent <[email protected]>

* MUL-6928 fix(issues): prefilter short contains needles too

The single-character skip rule was derived from the pg_trgm proxy used to
develop this change, where a 1-character needle can only drive a sequential or
full-index scan. pg_bigm is different — indexing 1- and 2-character keywords is
the capability it exists for over pg_trgm — so the rule denied the target
extension exactly the searches it was chosen to serve, and a single CJK
character is already a word.

Measured on PostgreSQL 16.15 with pg_bigm 1.2 over 1M issues: a rare
1-character CJK needle goes from a 134ms Parallel Seq Scan to a 1.2ms Bitmap
Index Scan (8.3 -> 10527 tps at 4 clients). Only the JSON-escaping guard
remains, which is about soundness rather than benefit.

Co-authored-by: multica-agent <[email protected]>

* MUL-6928 fix(issues): analyze the properties expression index after building it

Building an expression index does not collect statistics for its expression.
Until they exist the planner falls back to a pattern-length heuristic that
estimates any single-character LIKE at a few percent of the table, so it cannot
tell a needle matching 50 rows from one matching 50,000 — and migration 446's
index ends up built, valid, ready, and never chosen. Autovacuum does not rescue
a large, quiet issue table promptly: its default threshold is 10% of rows
changed.

Measured on PostgreSQL 16.15 with pg_bigm 1.2 over 1M issues, with
max_parallel_workers_per_gather=4: a rare single-character CJK needle plans as a
Parallel Seq Scan at ~101ms with 446 alone — indistinguishable from the
pre-change baseline — and as a 0.64ms Bitmap Index Scan once the statistics
exist. Whether the wrong estimate actually flips the plan depends on how cheap a
parallel scan looks, so without this the shipped behaviour would vary by
instance sizing.

447 is deliberately not gated on pg_bigm the way 446 is: ANALYZE is valid, cheap
(557ms on 1M rows) and side-effect-free everywhere, and refreshing ordinary
column statistics is useful even where the expression index was skipped.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-09-02 12:36:52 +08:00
4fe6586b32 MUL-6790 feat(issues): support operator matching in scalar property filters (#7765)
* feat(issues): support operator matching in scalar property filters

Closes #7692.

Scalar custom-property filters (text / number / date / url) previously
matched by exact equality only, so a text fragment or a numeric range
returned an empty list. A property filter member is now either a plain
string (exact equality — the unchanged default that keeps every existing
saved view, URL, and API call meaningful) or an operator object:

- contains — case-insensitive substring (text / url)
- gt / gte / lt / lte — numeric comparison (number)
- before / after — lexicographic comparison of the stored
  "YYYY-MM-DD" strings (date)

Backend: parsePropertiesFilterParam accepts {op, value} members and
compiles them to operator patterns; propertiesFilterPredicate renders
ILIKE / numeric / date-string comparisons and the static ListOpenIssues
unroll in issue.sql mirrors them. contains values are LIKE-escaped at
parse time and ops are fully validated before compilation, so the SQL
branches only ever see legal shapes. The table-filters request carries
members as raw JSON so both shapes survive the round-trip, and the
query fingerprint canonicalizes them byte-exactly.

Frontend: the filter menu grows an operator row above the scalar value
input ("is" is the default and still commits a bare string), the client
matcher in issueMatchesPropertyFilters mirrors the server predicates,
and chips summarize operator members with per-locale phrases or math
symbols. Operator members ride in saved-view baselines and chip deltas
through a canonical propertyFilterValueKey.

Verified: full Go suite (race) and the internal/handler property,
facet, and issue-table tests; new unit + DB-backed operator tests on
both tiers; pnpm typecheck and the core/views Vitest suites including
new operator matrix, menu, and saved-view baseline tests.

* test(issues): cover operator members through the table-rows endpoint

The properties filter serves through three paths — the paginated list,
the open_only static unroll, and the table rows/facets compile — but
only the first two had operator-member coverage. The table path has
PR-specific code (raw-JSON members in issueTableFiltersRequest and the
byte-exact sortedUniqueRawJSON fingerprint) that no test exercised.
Drive ListIssueTableRows with operator members ANDed across two
definitions so the decode → fingerprint → parsePropertiesFilterParam
chain is pinned like the other paths.

* fix(issues): canonicalize numeric operator bounds and align empty-needle contains

Adversarial review findings:

- strconv.ParseFloat accepts forms Postgres ::numeric rejects ("0x1p4"
  hex-float, "1_000" underscores on older PG). The dynamic predicate
  binds the parsed float and succeeds while the static open_only unroll
  casts the raw string inside SQL and 500s — same filter, two list
  endpoints, different outcomes. Canonicalize the compiled pattern
  value with strconv.FormatFloat(num, 'f', -1, 64) at validation time;
  the dynamic re-parse and the static cast now see plain decimals.
  Regression tests cover the hex-float bound on both serving paths.
- issueValueMatchesOperator treated an empty contains needle as
  match-all ("".includes("") is true) while the server rejects empty
  operator values — a hand-edited saved-view blob could match every
  issue client-side. The matcher now refuses empty needles, and the
  test asserts it against a SET value (the unset case short-circuits
  before the operator runs and could never catch this).
- Server e2e: pin unset-key exclusion for contains / before / after on
  both serving paths via the seeded empty issue.

* fix(issues): align scalar property operator semantics

* test(issues): cover boolean contains mismatch

* fix(i18n): use exclusive Japanese wording for date before/after filters

The `before` / `after` operators compare strictly (`<` / `>` on both the
server predicate and the client matcher), so a date equal to the bound is
never a match. Japanese 以前 / 以後 mean "on or before" / "on or after" —
they include the reference date, so a user filtering 「2026-03-01 以前」
reasonably expects March 1 in the results and does not get it.

より前 / より後 carry the exclusive meaning the implementation actually
has. English "before"/"after" and Chinese 早于/晚于 are already exclusive
and are left unchanged.

Korean 이전 / 이후 share the same Sino root and are inclusive by
dictionary definition, but they are also the conventional rendering for
before/after filters in Korean software — left as-is pending a native
speaker's call rather than guessed at.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-09-01 17:21:11 +08:00
438a30ef00 MUL-6902 fix(agent): fix Pi v0.4.37 Windows session regression (#7843)
* fix(agent): avoid locking Pi transcript on Windows

Co-authored-by: multica-agent <[email protected]>

* test(pi): cover sidecar lock contention and release

* test(ci): run the Windows Pi session-lock regression tests

The windows-execenv job runs ./pkg/agent through an explicit -run
allowlist, and the ubuntu backend job cannot compile a windows-tagged
file. Without this step the two tests added for #7840 would never
execute in CI, leaving the same coverage gap that let the v0.4.37
regression ship.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: worker-opencode <[email protected]>
Co-authored-by: multica-agent <[email protected]>
Co-authored-by: J <[email protected]>
2026-09-01 13:12:38 +08:00
mrlonelyandL 108407088c MUL-6836: fix(codex): allow explicit standard speed (#7756)
* fix(codex): allow explicit standard speed

* fix(codex): negotiate explicit standard support

---------

Co-authored-by: L <[email protected]>
2026-09-01 13:04:32 +08:00
f1ce9d4542 MUL-6883 perf(server): stop rescanning settled delegated-failure recoveries (#7820)
* feat(server): retire settled delegated-failure recoveries from the outbox scan

ListPendingDelegatedFailureRecoveries had no indexable notion of a finished
recovery. Its partial index (343) selected every recovery comment ever
written, so each sweeper tick re-proved through four joins and two NOT EXISTS
subqueries that history was already handled. LIMIT bounds the rows RETURNED,
never the rows CHECKED, so a tick that found nothing still grew more expensive
as history accumulated.

Add comment.recovery_settled_at and a partial index that holds only unsettled
recovery comments, and write the marker inside the transaction that makes a
task terminal. Every other exclusion the scan applies is reversible — an issue
can leave 'done', a cancelled retry can be superseded — so none of them can be
frozen into an index predicate. A delivery receipt on a terminal task cannot be
taken back, which is what makes it safe to record as durable state.

The marker is deliberately written only once the task is terminal.
SetTaskDeliveredCommentIDs replaces rather than appends a receipt while a task
is still dispatched, so a reclaim by a differently-capable daemon can drop an
id it will not deliver; settling at receipt-write time would freeze that
legitimate transient window into a permanently lost recovery. Exhaustion is the
one terminal outcome whose receipt lands on a row that may still be running, so
it retires the comment directly in the same transaction.

An unmarked row is the safe direction: it is scanned exactly as it was before
this column existed. That is what makes the rolling-deploy window harmless —
an older replica that writes a receipt without the marker only leaves work for
the backfill.

Co-authored-by: multica-agent <[email protected]>

* feat(server): backfill delegated-failure recoveries settled before the marker

The marker only covers tasks that reach a terminal status after the writing
code ships. History written before that keeps matching the unsettled index
predicate and keeps being re-checked on every sweeper tick, which is the cost
this change set exists to remove.

The walk retires a candidate only when a terminal task already recorded it in
delivered_comment_ids — the same monotonic condition the runtime path uses.
Its keyset watermark advances over every candidate EXAMINED rather than every
candidate marked: a recovery that is genuinely still pending must be left
alone, and keying on updated rows would hand back the same watermark forever
and never terminate.

Separate from migrations and startup, in the shape the other backfill commands
already use, so a large comment table cannot block a deploy. Running it is a
performance recovery step, not a correctness step — until it finishes the scan
behaves exactly as it does today.

Co-authored-by: multica-agent <[email protected]>

* fix(server): close the review gaps in the settled-recovery backfill and paths

Three fixes from review.

1. The backfill reproduced the bug it was written to fix. Its keyset ordered by
   id while idx_comment_delegated_failure_unsettled is (created_at, id), so the
   candidate scan read every unsettled row and sorted it to return LIMIT of
   them; and the receipt test ran a per-candidate EXISTS over agent_task_queue,
   the largest table in the database. The keyset now matches the index, and the
   receipt lookup is inverted into one agent_task_queue read per BATCH that
   hash-joins delivered ids back against the candidates.

   Measured on 500k comments / 200k tasks / 50k unsettled recoveries:
   17,393ms and 816,963 temp blocks before, 235ms and no temp spill after. The
   candidate scan goes from reading 50,000 rows to reading the 1,000 it needs.

   Total walk time is flat across batch size (50x228ms, 10x1016ms, 5x2199ms —
   all ~11s), so batch size only trades how long one statement holds row locks.
   1000 stays, now for a measured reason rather than a guessed one.

2. Bulk cancellation paths could strand a delivered receipt permanently
   unsettled, which lets the new index regrow the history it exists to remove.
   Manual rerun's pending-slot clear and agent archive were the two named in
   review; auditing the rest found comment-edit cancellation, both chat-session
   cancellations, workspace revoke and runtime teardown in the same position.
   All now settle — inside the caller's transaction where one exists, and
   through CaptureCancelledTasks for bulk paths that finalize outside one. The
   invariant is stated on the exported helper so the next such path has
   somewhere to read it.

3. --max-batches could not actually resume. The watermark lived only in the
   process, so a bounded rerun restarted at the front of the index — which is
   precisely where the rows this walk never settles live, so repeated bounded
   runs would rescan the same prefix and never reach the settleable history
   behind it. Adds --checkpoint-file (atomic replace, read on start) and
   explicit --after-created-at/--after-id, and logs the resume cursor on every
   exit path including SIGTERM.

Co-authored-by: multica-agent <[email protected]>

* fix(server): make delegated-failure settlement terminal-checked and atomic

Two fixes from review.

1. The settlement SQL trusted its caller. "A delivery receipt is only immutable
   once the task is terminal" is the safety premise of the whole design, but the
   exported helper enforced nothing: passing a dispatched or running task would
   settle a receipt a reclaiming daemon can still replace, and the marker is
   monotonic, so nothing could undo it. The predicate now lives in the SQL, so a
   non-terminal task updates no rows instead of destroying the obligation.

2. Bulk settlement was post-commit best-effort, and the comment claiming the
   sweeper would replay a failed one was simply wrong: the covering task is
   terminal and already holds the receipt, so ListPendingDelegatedFailureRecoveries
   excludes the comment. It would neither replay nor settle — it would sit in
   the partial index forever, restoring the unbounded scan this change set
   removes.

   Every terminal write now commits with its settlement. CancelTasksForIssue /
   ForAgent / ByTriggerComment and the rerun pending-slot clear run in a
   transaction; CancelQueuedChatTasks settles inside the one it already had; the
   sweeper and daemon-recovery bulk fails move behind TaskService wrappers that
   own the transaction; agent archive gets CancelTasksForArchivedAgent, which
   cancels and settles atomically while still skipping the per-task broadcast
   the archive event makes redundant.

   BroadcastCancelledTasks, CaptureCancelledTasks and HandleFailedTasks run
   after their caller committed, so they no longer settle at all — a failure
   there could be neither rolled back nor repaired. The best-effort helper is
   deleted rather than left available; every remaining call site passes a qtx.

Tests: a dispatched/running task with a receipt is not settled (fails without
the SQL predicate), a bulk cancel whose settlement fails rolls the cancel back,
and the archive path settles through its atomic entry point.

Co-authored-by: multica-agent <[email protected]>

* docs(server): correct settlement invariant comment and drop leaked test function

The SettleDeliveredDelegatedFailureRecoveries doc still described the
pre-fix contract — settlement reachable through CaptureCancelledTasks or a
post-commit best-effort call — which is exactly the shape the atomicity fix
removed. Restate the rule as it now stands (same qtx as the terminal write,
never from a post-commit helper) so the stale text cannot guide a future
change back into stranding rows in the partial index.

TestBulkCancelRollsBackWhenSettlementFails dropped its trigger but left
reject_recovery_settlement() behind in the shared test database; drop the
function too.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
Co-authored-by: Eve <[email protected]>
2026-09-01 12:11:50 +08:00
7c1d826a98 chore(metrics): remove database-sampled metrics (#7819)
Co-authored-by: Sol-Boy <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-31 17:06:17 +08:00
mrlonely 2ebe0ae327 MUL-6862: fix(codex): widen thread setup handshake budget (#7790)
* fix(codex): widen thread setup handshake budget

* fix(codex): preserve handshake timeout overrides
2026-08-31 17:02:24 +08:00
db86c75e12 MUL-6878: avoid per-row issue status resolution (#7814)
* perf(server): avoid per-row issue status resolution

Co-authored-by: multica-agent <[email protected]>

* fix(server): address issue status query review nits

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Sol-Boy <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-31 16:00:45 +08:00
Janez ŠtuparandJanez Stupar bfe32f5e39 fix(runtime): wake daemon for local skill requests (#7713)
Co-authored-by: Janez Stupar <[email protected]>
2026-08-31 15:31:29 +08:00
8d7e6f6a0a perf(server): resolve only the requested skill bundles (#7810)
The daemon resolves one skill bundle per request -- deliberately, so each
download gets its own size-scaled deadline and is cached independently
(GH #4505). ResolveTaskSkillBundles answered each of those by calling
LoadAgentSkillBundles, which reads and hashes EVERY skill the agent has,
then picks the single requested bundle out of the result and discards the
rest.

So the per-request cost scaled with the agent's skill count, and the
number of requests did too. A cold dispatch for an agent with N skills
read the whole skill set N+1 times and computed N*(N+1) bundle hashes to
deliver N bundles. #7689 cut each of those loads from N+1 queries to 2,
which fixed the round-trip count but not the multiplier: the bytes read
from Postgres and the SHA-256 work stayed quadratic, and because the
daemon issues these requests serially, that work lands directly in
prepare latency.

Load only what was asked for:

- ListAgentSkillsByIDs is ListAgentSkills narrowed to a set of ids. The
  junction predicate is unchanged, so it is still the authorization: an
  id the agent does not have enabled returns no row, which the caller
  reports as not-found. It is deliberately NOT workspace-scoped, because
  ListAgentSkills is not either -- a stricter resolve could 404 a ref
  that claim had just advertised, breaking prepare instead of protecting
  it.
- Files come from ListSkillFilesBySkillIDs, the batch query #7689 added,
  for those skills only.
- Builtins resolve by name from the embedded FS and touch no query.
- Only the requested skills are hashed.

For 15 skills / 600KB that takes a cold dispatch from ~9.6MB read and 240
hashes to ~1.2MB and 30, with no schema change, no new index, and no
change to the wire contract: stale-hash still returns the current bundle,
plugin hash pinning still conflicts, response order still follows the
request, and a read failure still fails closed with a 500 rather than a
404 the daemon would treat as permanent.

LoadAgentSkills and the new loader share one assembly helper, so claim
and resolve cannot drift on skill order, per-skill file order, or the nil
file list of a skill with no files.

Tests: an agent with three skills resolving one must issue one scoped
query for one id and never the full ListAgentSkills; a builtin ref must
touch no query at all; request order and ref dedup; and the denials the
old full-set lookup used to provide -- unassigned skill, disabled
assignment, another workspace's skill, unparseable id, a source with no
server-side producer -- plus a scoped-read failure returning 500. Both
the scoped load and the enabled predicate are mutation-verified.

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-31 15:14:18 +08:00
be93feb063 MUL-6792: perf(server): batch skill-file loads in LoadAgentSkills (#7689)
* perf(server): batch skill-file loads in LoadAgentSkills

LoadAgentSkills issued one ListSkillFiles query per skill inside a loop,
so an agent with N skills cost N+1 round trips on the task-claim hot
path (three callers in daemon.go). Each query also pulls full file
bodies, so the fixed per-query overhead (pool acquire, RTT, parse) was
paid N times.

Add ListSkillFilesBySkillIDs (skill_id = ANY(...), ordered by skill_id,
path — backed by idx_skill_file_skill) and load all files in one query,
grouping by skill_id in a single linear pass. N+1 becomes 2. Output
order is preserved by assembling in ListAgentSkills order. Total bytes
transferred are unchanged; this removes round trips and pool acquires,
not payload. The per-skill ListSkillFiles is retained for the single-
skill handler callers that legitimately need it.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(server): fail closed when the agent skill read fails

Batching the per-skill ListSkillFiles loop into one query changed what a
swallowed read error means. Before, one failed query dropped one skill's
supporting files. Now a single failure drops the files of EVERY skill --
and the `files, _ :=` discard let claim and resolve carry on as if that
were the agent's real skill set.

Nothing downstream can catch it. BuildAgentSkillBundles hashes whatever
loaded, so the refs sent at claim and the bundle served at resolve agree
with each other; the daemon's validateSkillBundle compares a bundle
against a ref derived from that same bundle, accepts it, and writes it to
the on-disk cache. The agent then starts on skills whose rules are
missing, and the next dispatch reuses the cached truncated copy. The same
was true of the ListAgentSkills error above it, which could silently
strip every workspace skill from an agent.

LoadAgentSkills and LoadAgentSkillBundles now return the error, and the
three call sites settle it:

- claim (both the slim skill-refs path and the legacy inline-skills
  path) refuses with 500 via rejectClaimSkillLoad and leaves the task
  dispatched, so the stale-dispatched reclaim redelivers it -- the same
  treatment rejectClaimSourceLoad gives a transient source read.
- ResolveTaskSkillBundles returns 500, which the daemon's existing
  resolve retry schedule already treats as transient, instead of serving
  a bundle built from a failed read.

Tests: service-level coverage for both read failures, for
LoadAgentSkillBundles failing closed rather than returning built-ins
only, and for the batch grouping itself -- the fixture puts the two
result sets in deliberately different orders (skills by name, files by
skill_id) so a positional zip would hand each skill another skill's
files, and asserts the per-skill query is no longer issued. Handler-level
coverage asserts the 500s and that a refused claim leaves the task
dispatched and unstarted.

Co-Authored-By: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Claude Fable 5 <[email protected]>
Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-31 14:52:00 +08:00
mrlonelyandL 6231962e98 MUL-6840 fix(pi): preserve sessions across workdir changes (#7760)
* fix(pi): resume session across workdir changes

* fix(pi): serialize session transcript writers

* fix(pi): keep busy sessions resumable

---------

Co-authored-by: L <[email protected]>
2026-08-31 13:49:38 +08:00
88e9170dc1 refactor(redact): drop home-directory masking from Text (#7772)
redact.Text rewrote the local home directory to /Users/****/ on its way
to the database and the WebSocket broadcast. Removing it.

It was never an access-control boundary. Anyone who can read a task
transcript already sees repository paths, file contents, commands, diffs
and error output; masking one path segment out of all that protected
nothing, while it did make paths non-copy-pasteable and harder to line up
when debugging a run.

It was also incoherent about whose home directory it hid. homeDir and
username were resolved from the process running Text, and the main
transcript fields (Content / Output) are only redacted in the server's
ingest handler -- so a hosted deployment matched them against the
*server's* home directory, never the machine actually running the agent.
Only tool inputs and stderr diagnostics went through the daemon-side call
where the resolved home was the right one.

Transcript visibility is an authorization concern and belongs at that
layer. Secret redaction (API keys, tokens, PEM blocks, connection
strings) is a real boundary and is untouched.

Two tests asserted the removed behaviour and now assert its absence, so
nobody reintroduces host-path masking as a side effect of another fix:
redact.TestTextLeavesHomePathIntact (replacing TestRedactHomeDirectory
and TestInputMapRedactsNestedHomePath) and the home-path arm of
agent.TestCursorExecuteReportsSanitizedStderrOnProcessFailure, which
keeps its bearer-token assertion.

Dropping the branch also resolves the Windows failure in
TestRedactHomeDirectory reported in #7744: Windows can return a
DOMAIN\user account name that is not a substring of the home path, so the
strings.Replace never fired. CI only runs Go tests on ubuntu, which is
why main stayed green.

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-31 13:39:39 +08:00
mikyan fe59262188 feat(agent): add Huawei Cloud CodeArts support (#6985)
* feat(agent): add Huawei Cloud CodeArts support

* fix(migrations): resolve CodeArts prefix collision

* fix(agent): address CodeArts review feedback

* fix(agent): tighten CodeArts integration scope

* fix(agent): resolve CodeArts follow-up review blockers

* fix(agent): redact CodeArts launch arguments

* docs: sync supported runtime counts

* fix(agent): own CodeArts model probe process tree
2026-08-31 12:30:59 +08:00
64ec7f5416 MUL-6737: fix(timeouts): close the two gaps the 2h inactivity budget left open (#7699)
* fix(timeouts): close the two gaps the 2h inactivity budget left open (MUL-6737)

Two independent places where a timeout still answered the wrong question.

Codex runs a semantic-inactivity timer inside the app-server protocol, and
unlike the daemon's watchdog it cannot see that a tool is in flight: a
commandExecution that prints nothing for the window trips it anyway. At a
fixed 10m ceiling that timer killed a quiet test suite or an
output-buffering `docker build` long before the 2h daemon budget meant to
protect it, which made "we raised the budget to 2h" false for Codex users.
It now derives from the daemon budget, taking the larger of the idle and
tool windows because it stands in for both. A tool budget of 0 means "never
force-stop during a tool", which this timer cannot express, so it falls back
to the idle budget rather than running unbounded; with the watchdog suite
disabled entirely Codex keeps its own built-in default, as it always has.
The env override and flag are unchanged.

Queued tasks expired on a pure wall clock: queued longer than a TTL meant
failed. That conflated "nobody is coming for this" with "the queue ahead is
long", and only the first is a failure. MUL-6558 was the second — a
self-hosted runtime with low concurrency held its own queue past 2h and
healthy work died as queued_expired. The TTL knob added then only moved the
cliff. Expiry now keys on whether the owning runtime still proves liveness,
the same signal FailTasksForOfflineRuntimes already uses for
dispatched/running rows, so a daemon going down retires its queued and its
in-flight work on one clock instead of two. A busy runtime keeps its
backlog for as long as it takes. MULTICA_TASK_QUEUED_TTL is removed with
the wall clock it configured.

Heartbeat age is read directly instead of gating on runtime.status='online'
so a row stuck at 'online' with a dead heartbeat still releases its queue,
and rows with no usable runtime binding — runtime_id IS NULL from before
migration 251, whose CHECK landed NOT VALID, or a runtime row that is gone —
are expired on sight rather than stranded forever by a JOIN that matches
nothing.

Tests: the queued sweep test now runs both phases against the same rows,
asserting the 5h-old task survives while its runtime heartbeats (the
MUL-6558 regression) and only expires once that runtime goes silent.
Assertions are scoped to the test's own task ids, because a runtime-keyed
sweep legitimately also collects queued rows other tests left on the shared
fixture runtime.

Co-authored-by: multica-agent <[email protected]>

* fix(sweeper): give a queued task its own reconnect grace before expiring it

Review catch: keying expiry on runtime liveness alone was too aggressive in
one direction. Enqueue binds a task to agent.runtime_id without checking
that the runtime is up (task.go CreateAgentTask call sites), so for a
runtime that has already been dark for longer than the grace, the liveness
clause is satisfied the moment a new task is created — assigning an issue
to a laptop closed overnight would fail it inside one 30s sweep tick
instead of waiting for the machine to come back. That is the opposite of
what "reconnect grace" promises.

Require the row's own age as well: a queued task now waits a full grace,
counted from when it started waiting, before it can be given up on. A
heartbeating runtime still never expires its backlog, so the MUL-6558 fix
is unchanged. The age bound also stops the sweep re-evaluating the runtime
subquery against every queued row on every tick.

Also remove the deploy surfaces the previous commit missed — the knob was
deleted from the server but helm (values + configmap), the self-host
compose file, and the CI-run helm render assertions still advertised it, so
operators would have kept setting a variable nothing reads. Same for five
orphaned comment lines in .env.example describing the removed variable.

Correct the fail-closed comment on the unbound-runtime arms: they are not
reachable for "rows predating migration 251" — runtime_id was NOT NULL from
migration 004 until 251 replaced it with a CHECK that every insert and
update is verified against, and the migration-004 FK is ON DELETE RESTRICT.
The arms stay as defence against a future schema change, which is what the
comment now says.

Tests: a third phase covers the reported case — a task enqueued against an
already-dead runtime survives, then expires once it has waited a full grace
of its own. Phase 2's expectation is corrected accordingly: the row created
at now() no longer expires when its runtime dies, because it has not waited
its own grace yet.

Co-authored-by: multica-agent <[email protected]>

* docs(timeouts): state both queued-expiry conditions and correct the FK claim

Review follow-up on b567511.

The user docs still described queued expiry as a single condition — "fails
once the runtime has been silent past the reconnect grace" — which is the
pre-b567511 behaviour and skips exactly the case that commit fixed. A
reader assigning work to a machine that went offline last night would
conclude the task already qualifies to fail. All four locales of tasks,
daemon-runtimes and troubleshooting now state both conditions and why the
second one exists.

The environment-variables pages contradicted themselves: the env table had
been updated to say the Codex semantic budget is derived, while the
`multica config` supported-keys table further down still advertised a 10m
default. That table now describes the resolution rule instead of a value.

The schema comment added in b567511 named the wrong constraint.
agent_task_queue_runtime_id_fkey (migration 004) is ON DELETE CASCADE; the
ON DELETE RESTRICT one is agent_runtime_id_fkey on agent.runtime_id. The
conclusion is unchanged — a queued row cannot reference a deleted runtime —
but the mechanism is a cascade plus migration 251's explicit unbind of
history rows, not a restrict.

Also log the resolved codex_semantic_inactivity at daemon startup, for the
same reason tool_watchdog is logged: it is now a derived value, so without
it the effective Codex budget cannot be read anywhere until a timeout
fires. And correct a test comment that still claimed both non-exempt rows
fail — the fresh row now stays queued, which is the assertion right below it.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-28 18:00:35 +08:00
Bohan Jiang dcc2d58251 MUL-6737: fix(daemon): raise the agent inactivity budget to 2h and derive the tool budget (#7685)
The idle watchdog was sized to notice a hang quickly (30m), but the two costs it trades between are not symmetric: force-stopping a healthy run throws away the work and flips the issue to blocked, while noticing a hang late only holds a concurrency slot.

- DefaultAgentIdleWatchdog 30m -> 2h, sized by the longest legitimate silent step.
- AgentToolWatchdog now derives from the idle budget in both directions; MULTICA_AGENT_TOOL_WATCHDOG still overrides and 0 keeps its 'never force-stop during a tool call' meaning.
- Watchdog tick capped at 5m, so worst-case detection is budget + 5m instead of budget * 1.5.
- Dropped the unreachable 30s tick floor (window >= 1m implies window/2 >= 30s) and asserted the property it was written for directly.
- Removed the dead MULTICA_CODEX_TIMEOUT from .env.example; documented the generic watchdog and Codex timeout rows in CLI_AND_DAEMON.md; synced four doc locales.
2026-08-28 16:08:08 +08:00
VvVandweiweiwei 7d128b5a45 fix(daemon): own the openclaw CLI process tree so its deadline is enforceable (MUL-5467, MUL-5630, MUL-6736) (#6275)
* fix(daemon): own the openclaw CLI process tree so its deadline is enforceable (MUL-5467)

openclawCLITimeout has carried a "Known gap (deliberately not fixed here)"
note since #6084: the 5s deadline could not actually bound an
`openclaw config ...` invocation. This closes it, and fixes a second
OpenClaw misbehaviour found alongside it.

## Bug 1 — the deadline was decorative

exec.CommandContext kills only the direct child. cmd.Output() blocks in
Wait() until the output pipes os/exec manages reach EOF, and cancelling the
context does not unblock that io.Copy. So when openclaw leaves a descendant
holding stdout — its `openclaw-config` helper, or cmd.exe → node on Windows
— the call runs for the *descendant's* lifetime.

#6084 measured this on linux/dash (a shim whose backgrounded child slept 6s
took 6.01s against a 150ms deadline), tried a cmd.WaitDelay backstop, and
reverted it on review because it bounds the call by leaving the descendant
running — trading a hang for a process leak that nothing on Unix reaps. The
note left behind names the real fix: process-tree ownership.

Three call sites had the shape:

  - execenv.execOpenclawCLI       on the task's critical path
                                  (Prepare → prepareOpenclawConfig)
  - agent.detectCLIVersion        runs for every provider inside the
                                  daemon's blocking registration preflight
  - agent.discoverOpenclawAgents  four invocations per call

## Bug 2 — the answer was already on stdout

Measured on an OpenClaw host (openclaw 2026.5.27):

  openclaw --version    258ms  exits cleanly
  openclaw config file    60s  correct path printed, then never exits
  openclaw agents list    60s  correct list printed, then never exits

Waiting for exit turns two working commands into a task-fatal error:

  agent_error.process_failure (prepare execution environment: execenv:
  prepare openclaw config: locate openclaw active config:
  openclaw config file: context deadline exceeded (process: signal: killed))

The contract of these commands is "print a value". Once the value has
arrived and nothing more is coming, whether the process tidies itself up is
not the caller's business, and it should not fail a chat task.

## The fix

Two helpers in server/pkg/agent, both built on the process-group primitives
codex/claude/opencode already use — no new platform-specific functions:

  - RunCollect: hands os/exec an *os.File instead of a buffer, so os/exec
    starts no copy goroutine and Wait returns the instant the direct child
    exits. Then signals the child's process group, which reaps the helper
    and releases the last write end, so our own readers see EOF. Unlike
    WaitDelay this reports the CLI's real exit status, which matters because
    a failed `--version` probe skips runtime registration entirely, and
    because openclawShimDiagnostic type-switches on *exec.ExitError.
  - RunCollectQuiet: returns when the process exits, or when stdout has
    produced output that then stayed idle for 400ms — whichever comes first.
    Reports quiet=true on the idle path so callers can log the CLI's failure
    to exit without failing on it.

The idle shortcut deliberately cannot mask a silent CLI: with no output
there is nothing to salvage, so the deadline still governs and the call
still fails. A non-zero exit still fails too, with stderr intact.

execOpenclawCLI keeps its shim diagnostics and its context-attribution
order byte-for-byte; only the exec mechanism changes.
discoverOpenclawAgents keeps its existing 30s function-level budget rather
than gaining a per-attempt one.

Windows: there is no process group to signal, so calls are bounded (by the
drain grace) but a descendant can still outlive one. Owning descendants
there needs a Job Object and is not attempted here.

## Verification

  - Both packages green with -race; 6 consecutive runs, no flakes.
  - All 51 existing Openclaw* tests in execenv pass, including
    TestExecOpenclawCLITimeoutIsNotMisdiagnosedAsMissingInterpreter and
    TestExecOpenclawCLICancellationIsWrapped — their shims produce no
    stdout, so the salvage path is not reachable and the
    errors.Is(err, context.DeadlineExceeded) contract is preserved.
  - 13 new tests. Both mechanisms are mutation-verified:
      * RunCollect reverted to buffers with no group reap → the guard test
        hangs and the dump is the production stack: cmd.Wait →
        awaitGoroutines blocked, writerDescriptor parked in io.Copy.
      * RunCollectQuiet's idle path disabled → both the unit test and the
        execenv end-to-end test fail after a full 60s.

* fix(daemon): address review on the openclaw CLI collector (MUL-5467)

Three blocking items from review of #6275. All three were reproduced before
being fixed, and the first one was a genuine correctness hole rather than a
style objection.

## 1. The deadline branch salvaged partial answers

RunCollectQuiet returned success from the ctx.Done() branch whenever stdout was
non-empty, without requiring the idle grace or any check that the answer was
finished. A CLI still streaming when the deadline arrived therefore had its
truncated output reported as a completed response. Reproduced with a stub
emitting a JSON document forever against a 250ms deadline: 9 runs in 10 came
back as success carrying an unparseable fragment.

The early return now needs two independent conditions, neither sufficient
alone:

  - an OutputComplete rule supplied by the caller accepts the buffer, and
  - the buffer has since been idle for idleGrace.

Reaching the deadline is never success. The captured bytes are still returned
alongside the error so a caller with its own rule can inspect them, but the
runner no longer decides on the caller's behalf. A nil rule disables the early
return entirely, which is the conservative default for a command whose output
shape has no completeness rule.

The rule cannot be inferred inside the runner, which is the point: only the
caller knows whether `{"agents":[{"id":"a"},` is truncated or legitimate.

  - `--json` subcommands -> agent.JSONOutputComplete (whole buffer parses).
  - `openclaw config file` -> openclawConfigPathComplete (last non-empty line
    looks like a path). Deliberately stricter than
    openclawParseActiveConfigPath, which resolves a relative line through
    filepath.Abs and would therefore accept a Doctor warning border as an
    answer. That matters because OpenClaw prints Doctor UI *before* the path
    (MUL-3136), so an early return on idle output alone hands back the banner
    as the config path — and nothing downstream catches it.
  - `agents list` text fallback -> RunCollect (wait for exit). Decorated text
    has no completeness rule and a silently short agent catalog is worse than a
    slow one; this path still gains the reaping and the bounded return.

## 2. quietWriter published bytes before their timestamp

The buffer write and the last-write timestamp were updated in separate critical
sections, so a reader could observe new bytes carrying a stale timestamp and
conclude the stream had gone quiet at the moment it was producing — truncating
an answer mid-write. Both now move in one critical section (outputBuffer).

## 3. The drain-timeout path raced the readers, and could park a goroutine

RunCollect read its buffers after the drain grace expired while io.Copy could
still be appending to them; the deferred Close on the read ends ran only after
the return values had been evaluated. Separately, RunCollectQuiet handed os/exec
io.Writers, so its cmd.Wait goroutine could sit forever on a descendant holding
the pipe — on Windows nothing released it, since there is no process group to
signal.

Both are now structural rather than patched. A single collector owns the pipes
for both helpers (the *os.File ownership RunCollect already had), and finish()
guarantees that when it returns, every goroutine startCollector spawned has been
joined and nothing can still append to the buffers: it reaps the tree, waits
bounded for Wait, and if the readers still have not seen EOF it closes the read
ends outright — which makes the in-flight Read return — and only then joins them.

## Verification

  - Both packages pass with -race under CI's flags (-p 2 -parallel 2).
  - All pre-existing Openclaw* tests still pass, including the timeout
    attribution and shim diagnostic ones.
  - New regression tests, all three named in the review: continuous output until
    the deadline (must not be success), a prompt followed by a delayed body (must
    not return the prompt), and a late non-zero exit within the grace (must be
    reported as the failure it is). Plus the completeness rules per subcommand,
    the banner-then-path case end to end through execOpenclawCLI, goroutine
    collection on both helpers, and the two outputBuffer contracts.
  - Windows contract coverage added (run_collect_windows_test.go) with a scoped
    step in the existing windows-execenv CI job, since the close-the-read-ends
    path is unreachable on Unix where the group kill releases the pipe.
  - Mutation-verified: restoring the unconditional salvage fails both the unit
    and the execenv regression; dropping the completeness rule from
    execOpenclawCLI makes the Doctor banner come back as the config path.

* test(daemon): make the openclaw config-path fixtures hermetic

The two execenv fixtures printed the literal path the host had printed,
`/root/.openclaw/openclaw.json`. openclawParseActiveConfigPath stats what it
parses and tolerates only os.ErrNotExist, so that path resolves differently by
environment: ENOENT on a developer machine, where /root does not exist, but
EACCES on a Linux CI runner, where it exists and is unreadable. The second case
is a hard error, so TestExecOpenclawCLIWaitsForThePathAfterDoctorBanner failed
in CI for a reason unrelated to the boundary it covers.

Both fixtures now print a real file inside t.TempDir(), which stats cleanly
everywhere. The second test only inspects raw stdout today, but a hard-coded
path there would be a trap for whoever next adds a parse to it.

The literal path stays in the completeness-rule table, which calls the
predicate directly and never stats, so it keeps documenting the shape the CLI
actually emits.

* fix(agent): own one-shot CLI process trees on Windows

* fix(daemon): support current OpenClaw config discovery

* Address review, and narrow this branch to the process-tree fix

Blocking items from the review:

1. The deadline branch no longer salvages. It returns an error unconditionally,
   and callers supply a per-subcommand completeness rule (openclawOutputComplete)
   that decides whether the captured bytes are a finished answer. Regression
   coverage for the three shapes named in the review: output continuing right up
   to the deadline, a banner ahead of the real answer, and a late non-zero exit.

   finish() now reports, rather than being wired to a field that was never
   assigned. Two cases do not converge and both cost the caller something it
   would otherwise never learn: a process still outstanding in Wait after the
   tree kill, and a descendant still holding the pipe when the read ends had to
   be force-closed, which can truncate trailing output. collectorError joins that
   with the command's own error, so nothing is displaced.

2. outputBuffer publishes bytes and the write timestamp inside one critical
   section, so a poller cannot see new bytes with a stale timestamp and conclude
   the stream went quiet while it was in fact producing.

3. The drain-timeout path closes the read ends and waits for both absorb loops
   before any snapshot is taken, so no io.Copy can be appending to a buffer the
   caller is about to read. Windows contract coverage runs as its own CI job.

   The goroutine guarantee is stated at its real strength instead of "no
   goroutine outlives the call, on any platform". Convergence holds whenever the
   tree kill takes. When it does not, one goroutine can stay parked in cmd.Wait
   on a process nothing can terminate; that residue is not removable here, so it
   is reported instead of asserted away. On Windows the same case extends the
   call rather than leaking, because an anonymous pipe is not pollable and
   closing the read end waits for an in-flight Read to release it.

Also from review:

- A JSON error envelope is valid JSON, so JSONOutputComplete accepts one and a
  CLI that prints `{"error": ...}` and then lingers past the idle grace hands
  back err == nil with the envelope in stdout — where the exit-status path never
  sees it. All three JSON readers now check for that explicitly. It matters most
  for `config get agents.list --json`, where the envelope is how a 2026.6+ host
  says the key moved to the registry: decoding it as data turned a graceful
  fallback into a failed preparation. For the object-target read the failure is
  silent, since the envelope would be sanitized and written into the task's
  snapshot as if it were the user's config.

- RunCollectCmd selects on ctx.Done(). It takes a command the caller built, so
  enforcement must not depend on that caller having used exec.CommandContext;
  the ordering matches RunCollectQuiet so the two entry points cannot report
  different error shapes for the same situation. startCollector drops its unused
  ctx parameter rather than implying it enforces one.

- checkOpenclawVersion keeps stderr in the parse. It replaced CombinedOutput(),
  and a build that prints its version there would otherwise fail the gate on
  empty output, which skips runtime registration entirely. extractVersionLine is
  bypassed on purpose: parseOpenclawVersion already scans the whole text, while
  picking the first line containing a semver lets unrelated stderr noise answer
  for it. The probe is bounded because the caller's ctx is the task's, and
  ExplainExecError is applied here too so an ENOEXEC shim is still named
  (MUL-6164).

- TestRunCollectQuietWithoutCompletenessRuleWaitsForExit synchronizes on a
  marker the CLI writes after its answer instead of a deadline. Its two
  assertions want opposite budgets, and the reviewer reproduced the resulting
  flake 3/3 under load. Waiting for the marker also proves more than a timeout
  could: that the runner was still blocked at a moment when the answer was
  already fully available.

Scope: the MCP work that shared this branch is reverted to main's behaviour and
will come back on its own, because it needs a discussion this one does not. It
changed which command supplies the resolved MCP settings, restructured the
include chain, and moved where the user's config bytes live — none of which the
deadline fix depends on. Preparation for a managed-MCP agent behaves exactly as
it does on main here.

$OPENCLAW_HOME path expansion stays: openclawConfigPathComplete calls
openclawHomeRest directly, so the completeness rule for `config file` cannot
tell whether that shape is a finished answer without it.

* fix(agent): converge the collector's tree kill and stop failing calls on cleanup (MUL-5467)

Review follow-up on #6275. All three findings share one root cause: whether the
caller got its answer depended on whether cleanup converged, and cleanup
convergence depended on a single kill(-pgid) landing.

- Retry the tree kill across collectReapWindow. One pass loses a descendant
  whose fork completes between the kill's enumeration of the process group and
  the signal's delivery — measured 3 misses in 10 runs of the forking stub, each
  leaving a `sleep 300` reparented to init and stalling the call until the drain
  budget expired. The window is deliberately short so it does not widen the
  pid-reuse window reapProcessTree already documents.

- Replace the 5s EOF budget with a 400ms settle. Once the direct child has
  exited, at most one pipe buffer of its output can still be unread — a child
  that had written more would have blocked in write() and could not have exited
  — so a short cap cannot truncate a well-behaved CLI's answer. What it cuts
  short is a surviving descendant's trailing output, which is not the answer.

- Stop reporting cleanup residue as a call error. finish() now logs and returns
  nothing: a probe whose output arrived and whose helper the kernel would not
  kill is exactly the case #6084's cmd.WaitDelay got wrong, and a failed
  version probe skips runtime registration entirely. This drops finishErr,
  collectorError and the drainForced/waitPending split, which the four call
  sites were interpreting two different ways by accident.

- Fix the reaping tests. They asserted on `$$` captured inside `( )`, which
  POSIX keeps at the invoking shell's pid, so they named the direct child that
  cmd.Wait reaps regardless — all of them passed with reapProcessTree removed,
  while the run leaked 7 orphans. Record `$!` instead, and add
  TestRunCollectRetriesTheTreeKill, which injects a missed pass because the race
  lives inside one syscall and cannot be forced from a stub.

Verified: collector group 0/10 failures (was 3/10) with 0 orphans, 0/5 under
-race, full ./pkg/agent -race green, execenv Openclaw -race green, and both
mutations — retry removed, reaping disabled — fail the tests that now cover
them, on their own assertions rather than on a cleanup error.

* refactor: move the `$OPENCLAW_HOME` expansion out of this branch

It is not process-tree work. `openclaw config file` printing
`$OPENCLAW_HOME\...` instead of the expanded path is the same class of bug as
#6630 — a path the daemon cannot stat, reported as a fresh install, wrapper
without the user's `$include` — and #6643 shipped the tilde half of it as its
own change. This half belongs there too, not folded into a branch about
owning the CLI's process tree.

Removed here: openclawHomeRest, its branch in expandOpenclawPath, the unit
test, and the completeness-rule row that depended on it. Neither PR now
depends on the other.

openclawConfigPathComplete therefore does not recognize the variable shape,
which is correct in isolation: `main` cannot expand it either, so a rule that
accepted the line would hand the caller a path the parser then fails on.
Such a host keeps `main`'s behaviour — no rule match means RunCollectQuiet
waits for exit — so it loses the hang tolerance and nothing else. The comment
on that function says so and names the change that should close it, because a
gap nobody wrote down is one a rebase re-opens silently.

* fix(daemon): teach the config-path rule the `$OPENCLAW_HOME` shape (#7310 hand-off)

#7310 landed the parser side of this, so the deferred half is now due — and it
was recorded in the code precisely so a rebase could not drop it silently.

While expandOpenclawPath could not resolve `$OPENCLAW_HOME\...`, this rule had
to refuse the shape: accepting it would have reported a finished answer and then
handed the caller a path the parser failed on. The cost was that a host with that
variable set lost the early return entirely and went back to waiting for the CLI
to exit — which is the hang this branch exists to survive. With the parser
fixed, the refusal was the only thing left standing between those hosts and the
tolerance.

Resolved through openclawHomeFromEnv rather than read raw from the environment.
The value may itself be a tilde path that upstream expands before printing, so
`filepath.IsAbs` on the raw string answers false for `OPENCLAW_HOME=~/svc` and
silently withholds the early return on exactly the configuration #7310's review
was about. Mutation-checked: the raw-env version fails the new
`tilde-valued OPENCLAW_HOME` row with `rule(...) = false, want true`.

TestOpenclawOutputCompleteRules becomes subtests so a row can carry the
environment it is about, and gains three: an absolute OPENCLAW_HOME, a
tilde-valued one, and an unset one — the last staying false, so a line with
nothing to resolve it against still waits for exit and lets the parser report
the real error instead of this rule guessing.

* docs(agent): site the collector against outputOwned, not cmd.Output()

#7531 moved every one-shot probe in this package onto launch.go's
runOwned / outputOwned / combinedOutputOwned, which makes three claims in
this branch stale rather than wrong-in-detail:

- RunCollect called itself "the safe replacement for cmd.Output()". There
  are no cmd.Output() call sites left in pkg/agent. The comparison a
  reader needs is against outputOwned, which owns the same process tree
  and differs only in who owns the pipes — so that is what the doc
  compares, including why the probeWaitDelay bound is the right default
  for the probes it serves and the wrong one here.
- RunCollect's path-taking form advertised itself as the migration entry
  point for "the remaining cmd.Output() call sites (models.go's catalog
  probes, codex.go, deveco_models.go, thinking.go)". #7531 migrated all
  four to the owned helpers. It stays as the form for a caller holding
  only a path, and for a probe that turns out to need pipe ownership.
- collector's doc now says outright which of the two properties launch.go
  cannot provide, since "os/exec owns the draining" now describes
  in-package code rather than a stdlib call nobody here makes.

Also adds the `if: needs.changes.outputs.backend == 'true'` guard the new
Windows CI step was missing. Every other step in that job has it; without
it the step ran on frontend-only changes.

* docs(agent): cite upstream for the hang tolerance, not just the timings

The measured "prints its answer and then never exits" for `config file` and
`agents list` came from one host on openclaw 2026.5.27, which invites the
obvious objection: this is an upstream bug, so why compensate for it here
rather than wait for a fix.

Read at openclaw v2026.7.1, it is not a bug in a build — it is the exit
model. `src/cli/one-shot-exit.ts` provides an explicit opt-in,
`requestExitAfterOneShotOutput`, which `runCli` flushes after both streams
drain. In the shipped 2026.7.1 tarball exactly three modules take it:
`models list`, `models status`, and the hooks CLI. The commands this
package invokes — `config file`, `config get`, `agents list` — do not:
their success paths only print, and `runtime.exit` appears solely on their
failure paths, so termination is whatever the event loop happens to do.
A version that exits promptly is not promising the next one will.

Also pins openclawHomeRest's citation forward. `runConfigFile` still
prints shortenHomePath at v2026.7.1, and resolveHomeDisplayPrefix there
still selects `$OPENCLAW_HOME` on a non-empty trimmed variable and `~`
otherwise. Worth recording that the separator comes from the original path
rather than being inserted by the prefix, since that is what the rest-slice
in this function relies on.

* fix(daemon): size the config-file idle grace to the real warning gap

E2E on a real OpenClaw host (2026.7.1-2, plus 2026.6.8 and 2026.7.2-beta.7)
falsified two claims this branch was making. Both are corrected here; one
needed a code change.

**The 400ms idle grace was sized against the wrong gap.** The claim was
that a one-shot response arrives in back-to-back writes, so 400ms is
"orders of magnitude longer than the gap between those writes". The
first half is confirmed — 0.012-0.033ms between writes. The gap that
matters is a different one: with a config that produces Doctor and plugin
warnings, `openclaw config file` spread its stdout over 3289ms, of which a
single 3285ms silence sat between the end of the warning block and the
path. The default grace is eligible to fire inside that silence, on
whatever the warnings left in the buffer.

Why it lands on `config file` alone: upstream routes incidental console
output to stderr when argv carries `--json`
(withConsoleLogsRoutedToStderrForJson, src/cli/json-output-mode.ts at
v2026.7.1), so every `--json` caller here gets a stdout that is the JSON
document and nothing else, and a partial document cannot parse however long
the writer pauses. `config file` has no `--json`, so the warnings share its
stdout with the answer.

openclawConfigPathComplete is still the guarantee — a box-drawing border is
not an absolute path — and a grace can only be defence in depth, since the
gap is a property of the host's plugin and Doctor work. But it should not be
defence that loses by three seconds, so openclawQuietIdleGrace gives
`config file` 5s and leaves `--json` on the default. The cost is paid only
by a CLI that hangs: `config file` then returns in path + 5s rather than
path + 400ms, against a 30s deadline.

**RunCollect guarantee 2 overstated what a group kill can reach.** The
helper was measured holding the same stdout pipe as its parent (both
`pipe:[135202]`) while running with its own PGID and SID. `kill(-pgid)`
never reaches that, and neither does #7531's ownership, which uses the same
group. "Invoking a CLI on a timer cannot accumulate orphans" was therefore
wrong on Unix, and the three comments that repeated it are corrected. What
holds regardless is guarantee 1: it is pipe ownership, not the kill, that
makes Wait return and the deadline enforceable. Windows is unaffected — a
Job Object owns the tree irrespective of process groups.

Tests: TestOpenclawQuietIdleGraceCoversTheWarningGap pins the per-command
grace and asserts the `config file` value exceeds the measured gap;
TestExecOpenclawCLISurvivesTheWarningGapBeforeThePath drives a stub with a
warning block, a gap past the default grace, then the path, and requires
the path back.

* fix(agent): make pipe EOF the completion signal, and bound what is kept

Second review round found three blockers on b9b39eaea. All three reproduce
exactly as reported, and all three are cases where this branch was *worse*
than the launch.go helpers it replaced — which is the bar a replacement has
to clear, so the fix narrows the branch rather than patching around them.

**1. Leader exit was treated as answer completion (worst of the three).**
RunCollect*Cmd reaped the process group as soon as the direct child exited.
A wrapper can exit 0 while the real CLI — its descendant, holding the
inherited stdout — has not printed yet, which is the normal shape for an npm
or PowerShell shim. Measured here: the wrapper exits 0, its child prints the
version 500ms later, and this head returned an empty version with a *nil
error* in 427ms where outputOwned returns the version in 517ms. Empty-and-nil
is the worst available shape, and detectCLIVersion routes all 23 providers
through it.

Pipe EOF is the only signal that means "no more output is coming", so:

- detectCLIVersion, checkOpenclawVersion and discoverOpenclawAgents go back
  to launch.go's outputOwned / combinedOutputOwned, i.e. back to os/exec's
  own EOF wait. What this branch still fixes there is the *reporting*: a
  lingering descendant makes Wait report exec.ErrWaitDelay over an answer
  that arrived, and a failed --version probe skips runtime registration
  entirely (#6084 reverted a WaitDelay backstop over exactly that). So
  salvageProbeAnswer drops the error when, and only when, the caller's own
  parse found the answer — never on a non-zero exit, never on a deadline.
- The collector keeps its reason to exist (execenv's task-critical path) and
  gains awaitOutputAfterExit: after the direct child exits it waits for pipe
  EOF, bounded by collectDrainGrace (2s, matching probeWaitDelay), unless the
  caller's completeness rule says the answer is already in — so a CLI that
  prints and exits with a helper on the pipe still pays nothing.
- RunCollect / RunCollectCmd are deleted. With the probes back on
  outputOwned they had no production caller, and an exported wait-for-exit
  collector is exactly the API that invited this bug.

**2. A path-shaped warning could be returned as the config path.**
openclawConfigPathComplete accepted any absolute-looking last line, and the
previous commit's answer — a 5s grace — was the wrong dimension, as the
review pointed out by quoting this branch's own comment that no fixed grace
can bound the host's plugin work. Reproduced: a CLI printing an existing
plugin-cache.json path, pausing 6s, then printing the real path returned the
warning path after 5.51s.

Fixed by not asking a question content cannot answer. `openclaw config
validate --json` reports the path in a named `path` field, on every branch
(valid / invalid / file-not-found / error), in every version from v2026.5.5
— which is minOpenclawVersion — through v2026.8.1-beta.3, and `--json` makes
upstream route incidental output to stderr so stdout is the document alone.
openclawActiveConfigPath asks that first and keeps `config file` as a
fallback with no completeness rule at all: it waits for exit and fails closed
on the deadline. Upstream runs the same readConfigFileSnapshot() for both
commands, so preferring the JSON form costs no extra work; it costs one extra
invocation only on a CLI too old for either, which is pinned in
TestPrepareOpenclawConfigFallsBackWhenConfigFileUnsupported.

openclawConfigPathComplete, openclawQuietIdleGrace and
openclawConfigFileIdleGrace are gone. `--json` is now the only shape with a
rule, and the doc says why that is the design and not a gap.

**3. Retention was unbounded.** absorb appended without limit, where
outputOwned keeps the last probeStderrSampleBytes. Measured: 13,107,400 bytes
of stderr retained from a CLI writing continuously, ~400x the bound it
replaced. stderr now keeps its tail at collectStderrTail (= 32 KiB, so moving
a call site between mechanisms cannot change what a failed probe reports) and
stdout is capped at collectStdoutLimit and reports overflow as an error —
deliberately asymmetric, because a head-truncated answer is how a partial
catalog becomes a confident empty one.

Every new guard was mutation-verified by restoring the mechanism it covers:

| Mutation | Test | Failure |
|---|---|---|
| collector + reap on leader exit | TestDetectCLIVersionWaitsForAWrapperDescendant | `version = ""` in 0.42s |
| awaitOutputAfterExit returns immediately | TestRunCollectQuietWaitsForAWrapperDescendant | `stdout = ""` |
| drop the rule short-circuit | TestRunCollectQuietDoesNotWaitWhenTheAnswerIsIn | pays the full 2s |
| unbounded buffers | TestCollectedStderrKeepsOnlyItsTail | retained 13,107,417 bytes |
| unbounded buffers | TestCollectedStdoutOverflowIsReportedNotTruncated | 524,296 bytes, nil error |
| shape rule for `config file` | TestOpenclawActiveConfigPathIgnoresAPathShapedWarning | returns plugin-cache.json |

The new fallback test also caught a bug in this commit before it shipped:
openclawValidatedConfigPath accepted upstream's `CONFIG_PATH ?? "openclaw.json"`
literal, because expandOpenclawPath turned that relative name into a confident
path under the daemon's cwd. It now requires the reported value to be absolute
before expansion, allowing only the `~` and `$OPENCLAW_HOME` forms.

* fix(agent): repair the Windows-only collector test call sites

The mechanical conversion of the deleted RunCollect call sites over-widened
three tuples in run_collect_windows_test.go, which already used
RunCollectQuiet's four results. Host `go build` and `go vet` never compiled
them — the file is `//go:build windows` — so the break surfaced on the
windows-execenv job as `assignment mismatch: 5 variables but RunCollectQuiet
returns 4 values`.

Verified this time the way it should have been the first time:
`GOOS=windows go build ./...` and `GOOS=windows go vet ./...`, plus darwin and
linux. That is the check to run whenever this package's exported signatures
move, since a third of its process-lifecycle tests only compile on Windows.

* test(execenv): pin the config-path payload shapes E2E measured

E2E on OpenClaw 2026.7.1-2 confirmed `config validate --json` as the config-path
source and corrected two things this branch had inferred from source.

**The exit status is load-bearing in the wrong direction, and this was untested.**
Two of the three real states exit *non-zero* while carrying the answer: a missing
config file gives `{"valid":false,"path":"…","error":"file not found"}` and an
invalid one `{"valid":false,"path":"…","issues":[…]}`, both exit 1 with stderr
empty. A fresh install is the missing-file case, so reading a non-zero exit as
"no answer" would break first run. openclawValidatedConfigPath already ignored
the status, but nothing pinned it — writeOpenclawConfigStub could only exit 0, so
the tests covered the rarest branch. It now takes an exit code, and
TestOpenclawActiveConfigPathReadsThePathFromANonZeroExit drives all three
measured payloads.

**Two source-based claims corrected.**

- "Preferring the JSON form costs nothing extra" understated it. Measured over
  three runs each on the same host and config: 1.65/1.66/1.65s for
  `config validate --json` against 6.36/4.16/4.06s for `config file`. The JSON
  form is ~2.5-4x faster, which tracks the output it does not render — 715 bytes
  of JSON against 2663 bytes and 28 lines of warning UI.
- The reason a `--json` stdout is clean was attributed to upstream's
  console-log reroute (`withConsoleLogsRoutedToStderrForJson`). That function
  exists, but it is not what was observed doing the work: the Doctor and plugin
  warnings arrive as a `warnings` array *inside* the payload. The distinction
  matters for why JSONOutputComplete is a real signal — the stream is
  structurally one document, not merely quieter — so the comments now describe
  the measurement rather than the mechanism inferred from it.

Unrelated and pre-existing: `./pkg/agent`'s ACP timing tests (1-6s wall-clock
budgets) flake under load, and which ones fail varies per run. Control run of
`origin/main`'s whole package on this machine, same conditions: fails
TestCodexExecuteSemanticInactivityAllowsContinuousMessages and
TestCodexExecuteLegacyFirstTurnMessageSatisfiesProgress. Not introduced here.

* fix(agent): require a recognised version before salvaging a probe answer

The ErrWaitDelay salvage in detectCLIVersion was gated on `version != ""`,
and extractVersionLine falls back to the trimmed raw output when no line
carries a semver token — so any non-empty text satisfied it, including
whatever a wrapper printed before the real version existed.

Review reproduced it with a stub that prints a banner, exits 0, and has a
descendant print the version past the 2s WaitDelay: detectCLIVersion
returned "initializing plugins" with a nil error, and logged "CLI answered
but left its output pipes open" — the opposite of what happened. That value
is then persisted as the runtime version, for every provider, since
DetectVersion routes all 23 through here.

extractVersionLine now reports whether the semver scan actually matched, and
only that answers the salvage question. The trimmed-raw fallback still
stands on the success path: a CLI that exits 0 with an unusual version
format is reporting its version (#2516), and it is only unusable as evidence
that a bounded, incomplete read already holds the answer.

Mutation-checked: with the old gate,
TestDetectCLIVersionDoesNotSalvageABannerAsTheVersion fails with exactly the
reviewed shape — version="initializing plugins", nil error, misleading warn.

* test(agent): derive the collected-stdout cap from the largest answer

collectStdoutLimit argued for why a fail-closed cap is needed but not for
its value, and where it falls is a product compatibility boundary rather
than only a memory bound: a legal config past the cap stops a task from
starting at all.

The derivation is now recorded and executable. Measured answers on 2026.7.1-2
are 715 B (`config validate --json`) and 2663 B (`config file`); the largest
answer any call site asks for is the resolved config from `config get --json`,
which upstream puts no ceiling on. So the worst case is constructed — 1000
agents and 250 MCP servers with the fields OpenClaw carries, 547,291 bytes,
already orders of magnitude past a real deployment — and 8 MiB leaves 15x on
top of it while matching what the daemon already holds for a comparable
per-task payload (maxLocalSkillBundleSize).

Two tests pin it: the constructed high-water mark must round-trip through the
collector byte-identically with the stated headroom, and the boundary itself
is exact — at the limit the answer is returned whole, one byte past it fails
closed with errCollectStdoutTooLarge.

* fix(execenv): share one CLI deadline across both path-resolution attempts

openclawActiveConfigPath asks `config validate --json` first and falls back
to `config file`, and the fallback carried a fresh full deadline. That made
the worst case five budgets against a ceiling derived from four: at the 60s
override ceiling, 5m of CLI time alone, landing exactly on
daemon.defaultTaskPrepareTimeout — so the specific, non-retryable
ErrOpenclawCLITimeout collapses back into the generic retryable prepare
timeout, which is the outcome openclawCLIMaxTimeout exists to prevent.

Both invocations answer the same question, so they now share one context.
The multiplier is renamed to openclawMaxCLIDeadlinesPerPreparation because
that is what it always measured — the budget is per step, not per invocation.

The consequence is deliberate and asserted: a validate that burns the whole
budget leaves the fallback none, os/exec reports the expired context on Start,
and the failure still carries ErrOpenclawCLITimeout. A CLI that cannot say
where its config is within the entire deadline will not answer the same
question on a fresh one.

TestPrepareOpenclawConfigWorstCaseCLIBudgets now drives the real worst case
(validate fails, config file answers, registry fallback, managed MCP) and
counts distinct ctx deadlines rather than calls — the old test synthesized a
validate success from the `config file` response, so the two never fired in
the same run. Mutation-checked: with separate deadlines it reports 5 budgets
against a constant of 4.

---------

Co-authored-by: weiweiwei <[email protected]>
2026-08-28 14:04:10 +08:00
08a1e6e934 MUL-6742: fix archived-agent runtime garbage collection (#7641)
* fix(runtime): garbage collect archived-agent runtimes

Co-authored-by: multica-agent <[email protected]>

* fix(runtime): address GC review feedback

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Sol-Boy <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-28 12:49:08 +08:00
LinYushen 5fa65bd125 feat(billing): enforce Cloud issue count limits (#7619)
* feat(billing): enforce Cloud issue count limits

* fix(billing): address issue limit review feedback

* test(billing): exercise validation as manager

* fix(billing): resolve second review feedback

* fix(core): preserve entitlement limit discrimination
2026-08-27 14:23:09 +08:00
09a2410e88 refactor(agents): rename starter_prompts to conversation_starters (MUL-6709) (#7592)
* refactor(agents): rename starter_prompts to conversation_starters

The UI, docs and all four locales call this feature "Conversation starters"
(#7573), but every identifier behind it still said `starter_prompts` — the DB
column, the wire field, the `/api/config` capability flag, the TS types, the
i18n keys, and the `?focus=` token that a user can read in their own address
bar. The old name is also the one #7573 rejected on the way in: it promises a
prompt, while selecting a starter only prefills the composer.

Rename it everywhere so the identifier matches the product.

This is a breaking rename with no compatibility shim, and it is free exactly
once: the column arrived in migration 404 (#7261) after the last release tag,
so nothing has shipped. No deployed client reads the old name, no stored data
uses it, and `agent_conversation_starters_supported` has no version skew to
straddle. After this release, the same change would cost a wire alias, a
dual-written capability flag, and older Desktop/Mobile builds silently hiding
the editor.

Migration 404 stays as-shipped; 432 renames the column and its CHECK
constraint, so an already-migrated dev or staging database converges with a
fresh one.

Co-authored-by: multica-agent <[email protected]>

* docs(skills): name the conversation-starters surface for agents

`multica-creating-agents` mentioned `conversation_starters` only as a wire
field in two tables. It never said what the feature is called in the product,
where a human configures it, or what selecting one does — so an agent reading
this skill would tell a user about "conversation starters" as a JSON field and
leave them with nowhere to go.

Add the product name (both languages), the Instructions-tab deep link, and the
behaviour that makes the name accurate: selecting one only fills the composer.
Also record that an agent with none configured still shows three built-in
defaults, so "the Chat shows suggestions" is not evidence this agent has any.

Document one gap the field table implied away: the accepted-body list is not a
CLI surface. `conversation_starters` has no `agent create` / `agent update`
flag — only `agent copy` carries an existing value forward — so an agent
following the old text would build a create command that silently drops it.

Co-authored-by: multica-agent <[email protected]>

* fix(onboarding): keep the released starter_prompt field on the pre-v3 shim

The rename in ec44cc0ef swept up `starter_prompt` (singular) on
`bootstrapOnboardingRuntimeRequest`. That is a different contract from the
per-agent `starter_prompts` this PR renames: it is a field on the endpoint
marked "DEPRECATED, kept for desktop < v3", and unlike the per-agent one it
HAS shipped — it is in v0.4.34, released since fbd965e5bf.

Renaming it breaks the clients it exists for, and breaks them silently: the
handler decodes with a plain json.Decoder, so an old Desktop build's
`starter_prompt` would be dropped as an unknown field, the request would still
return 200, and the onboarding issue would quietly fall back to the default
description instead of the text the user typed. The test change hid it by
moving the fixture to the new name, removing the only coverage of the old one.

Revert both files. The two features share a word root and nothing else — the
free-rename argument for the per-agent field does not extend to a contract
that is already in the wild.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-26 19:01:01 +08:00
LinYushen 3b1018e9c9 Revert "MUL-6016: feat(task): retry exact Codex capacity failures (#6747)" (#7586)
This reverts commit 8fda4f22d5.
2026-08-26 18:05:17 +08:00
bbba9c5460 MUL-6697: enforce runtime access on task claims (#7571)
* fix(tasks): enforce runtime access on claims

Co-authored-by: multica-agent <[email protected]>

* fix(tasks): settle invalid runtime claims explicitly

Co-authored-by: multica-agent <[email protected]>

* test(tasks): cover runtime owner claim mismatch

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Sol-Boy <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-26 15:55:05 +08:00
54027ba763 MUL-6639: fix(skills): metadata-only skill listings and stall-based timeouts (#7574)
* MUL-6639: feat(skills): list skill files as metadata, not content

`multica skill get` and `multica skill files list` both downloaded every file
body to render output that shows none of them — `skill get`'s table view prints
four columns, and the file listing prints paths and timestamps. Past a few
hundred KB that stopped being waste and became a hard failure: a 599KB skill
could not be fetched at all, and the listing that would have shown which file
was oversized required downloading all of them first, so the diagnostic died
with the thing it was meant to diagnose (GH #7498).

Both endpoints now take `?include=content|metadata`. The metadata shape carries
path, byte size and content hash per file, plus the size and hash of the
SKILL.md body — computed in Postgres, so the bodies never leave the database.
Response size now tracks file count, not content size.

`GET /api/skills/{id}/files` defaults to metadata: it is a list endpoint, the
CLI is its only caller, and `GET /api/skills` already dropped content for this
exact reason (GH #2174). `GET /api/skills/{id}` keeps returning content by
default — installed web and desktop builds call it for the skill editor and
cannot be retrofitted with a query parameter. The CLI asks for metadata on both
and takes `--with-content` when the bodies are actually wanted.

The file listing gains a SIZE column, which is what makes an oversized skill
diagnosable without downloading it.

Co-authored-by: multica-agent <[email protected]>

* MUL-6639: fix(cli): fail skill transfers on stall, not elapsed time

`http.Client.Timeout` is a wall clock on the whole request, body included, so
it punishes exactly the transfer that is working: a 599KB skill arriving
steadily over a slow link was cut off at 30s mid-body, while a genuinely dead
connection was held open for the same 30s (GH #7498). Raising the constant
moves the cliff without removing it — the honest test is not "has this taken
too long?" but "is it still making progress?".

internal/cli/stall.go adds a client that asks the second question: bounded
connect, TLS and response-header phases, and a response body that fails after
15s with no new bytes. The wall clock stays as a loose 10-minute backstop —
demoted, not deleted, because dropping it to 0 would have left a stalled
request upload with no deadline at all.

Failures now say which thing went wrong. A new KindNetworkStalled reports "no
data received for 15s after N bytes" rather than a timeout, because only one of
those is fixed by waiting longer. Body-read errors are classified at all now:
wrapTransport only ever saw errors from http.Client.Do, which returns once the
headers arrive, so the failure #7498 actually reports reached users as a raw
"context deadline exceeded ... while reading body" out of the JSON decoder.

MULTICA_HTTP_STALL_TIMEOUT sets the no-progress budget. An explicitly set
MULTICA_HTTP_TIMEOUT still applies on this path, keeping its plain meaning —
the longest you will wait for a server — instead of silently reverting to a
total-elapsed limit.

Adoption starts with the skill commands, where the failure was reported; the
mechanism itself is not skill-specific. Graduation is one edit: point
NewAPIClient at NewStallAwareHTTPClient, fold StallAwareContext into
APIContext, drop newSkillAPIClient. A permanent second client would leave the
CLI with two timeout personalities, which is worse than either alone.

Co-authored-by: multica-agent <[email protected]>

* MUL-6639: fix(skills): hash raw UTF-8, keep the no-include default at content

Review found two problems with the previous two commits.

**The file hash was not over the file's bytes.** `sha256(content::bytea)` runs
the bytea *input* parser over the text rather than taking its bytes, so it
reads backslash escapes: a file containing `\x41` hashed as the single byte
`A`, disagreeing silently with the Go-side hash of the SKILL.md body.

Worse than the mismatch, and not caught earlier because the first check used
accented text with no backslash in it: `content::bytea` *errors* on a bare
backslash. `C:\Users\...`, a regex `\d+`, a LaTeX snippet — ordinary contents
of a skill's supporting files — fail with "invalid input syntax for type
bytea", making the listing 500 on the skills most likely to need it.
`convert_to(content, 'UTF8')` takes the bytes, and matches Go exactly.

**`/api/skills/{id}/files` must not change what a bare request returns.** The
previous commit flipped its default to metadata on the grounds that the CLI is
its only caller. That was the wrong reading of "only caller": the CLI is
installed software too, and an older `skill files list --output json` reads
`content`. It is the same argument that kept the detail endpoint's default —
applied inconsistently to a client that upgrades separately from the server.

Both endpoints now return content when `?include=` is absent. The shrink
travels with the caller instead: the CLI sends `include=metadata` itself, so
GH #7498 is fixed with no synchronized release, and the default can flip once
clients that send `include=content` have aged in. The default is now uniform,
so resolveSkillInclude no longer takes a per-endpoint parameter.

Tests: TestSkillFileHashCoversRawUTF8Bytes covers `\x41`, invalid hex, bare
backslashes, Unicode and empty bodies against the Go hash — verified to fail on
the previous SQL. TestSkillEndpointsWithoutIncludeStillReturnContent pins the
compatibility guarantee on both endpoints.

Not addressed, and noted as follow-up: the metadata path still loads the full
SKILL.md into Go, because loadSkillForUser needs the row for its tenant check.
The HTTP response no longer grows with content; the database read still does.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-26 15:45:06 +08:00
Roman Yufimov 620f99d8ce fix(codex): wait for authoritative turn completion (#7554) 2026-08-26 14:33:08 +08:00
8fda4f22d5 MUL-6016: feat(task): retry exact Codex capacity failures (#6747)
* docs: record Codex capacity retry design

* docs: add Codex capacity retry implementation plan

* fix(taskfailure): recognize exact Codex capacity error

* feat(task): retry exact Codex capacity failures

* feat(server): configure Codex capacity retries

* docs: expose Codex capacity retry setting

* fix(task): align capacity retry comments

---------

Co-authored-by: cuijs <[email protected]>
Co-authored-by: Claude <[email protected]>
2026-08-26 12:50:20 +08:00
LinYushen baf1bbf340 fix(billing): make Cloud seat enforcement automatic (#7542)
Replace the separate Fleet/capacity URLs and seat-capacity switches with the single managed MULTICA_CLOUD_URL connection. Strict seat admission turns on whenever Multica is Cloud-connected and fails closed when the capacity machine token is absent or invalid; the recovery worker starts only for a valid executor so live intents keep their retry budget during credential outages.

Split invitation throttling into a non-consuming precheck and post-reservation consumption. Persistent capacity_full and capacity_overcommitted rejections charge only the actor budget; transient failures charge none. Preserve every Cloud/proxy HTTP 429 as retryable with both Retry-After forms, defer outbox rows without spending attempts, and use Cloud's rate-limit scope so a workspace 429 delays only that tenant while global or unscoped 429s stop the current batch.

BREAKING: delete/unset MULTICA_FLEET_URL, MULTICA_CLOUD_FLEET_URL, MULTICA_CLOUD_FLEET_TIMEOUT, MULTICA_SUBSCRIPTION_CAPACITY_ENABLED, MULTICA_SUBSCRIPTION_CAPACITY_URL and MULTICA_SUBSCRIPTION_CAPACITY_WORKER_ENABLED before rollout. The startup guard rejects any non-empty value, so MULTICA_SUBSCRIPTION_CAPACITY_ENABLED=false still blocks boot. Configure MULTICA_CLOUD_URL, MULTICA_CLOUD_TIMEOUT and MULTICA_SUBSCRIPTION_CAPACITY_SERVICE_TOKEN instead; entitlements keep their independent MULTICA_ENTITLEMENT_POLICY_* configuration.

Companion: multica-ai/multica-cloud#57
2026-08-26 12:42:42 +08:00
8442504201 fix(runtimes): hide private runtimes from non-owners (#7435)
* fix(runtimes): enforce private runtime access

* fix(runtimes): gate agent model discovery

* refactor(runtimes): reuse isRuntimeUsableForUser on runtime detail

The detail page re-derived the owner-or-public rule inline instead of
calling the shared helper every other surface in this PR uses. The two
disagreed on an ownerless runtime: the helper is fail-closed (matching
the server's read gate, which reuses canUseRuntimeForAgent), while the
inline copy rendered Usage for a public one and could only 404.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: m2keloop <[email protected]>
Co-authored-by: Bohan-J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-26 01:48:44 +08:00
6cbb50378f MUL-6680: fix autopilot list subscribers + make CLI triggers discoverable (#7552)
* fix(autopilot): populate subscribers in the list projection (MUL-6680)

GET /api/autopilots passed a nil subscriber slice to avoid an N+1, but
AutopilotResponse.Subscribers has no omitempty and autopilotToResponse
builds the slice with make(), so a nil input serialized as "subscribers":
[] rather than being omitted. Every listed autopilot therefore claimed to
have no subscribers while GET /api/autopilots/{id} reported the real ones
for the same row at the same moment.

A silently wrong value is harder to work around than a missing one: the
key is present and reads as authoritative — the field's own doc comment
promises exactly that — so callers cannot tell "none configured" from
"not fetched". It was reported after costing a real misdiagnosis
("subscribers keep being cleared").

The N+1 premise was wrong: autopilot_subscriber's primary key leads with
autopilot_id, so one ANY($1) batch is index-supported and needs no new
index. The batch query mirrors the single-autopilot query's member join
and ordering exactly, so list and detail cannot drift apart on who counts
as a subscriber.

Reported at https://github.com/multica-ai/multica/issues/7549

Co-authored-by: multica-agent <[email protected]>

* feat(cli): make autopilot triggers discoverable (MUL-6680)

trigger-update, trigger-delete and trigger-rotate-url all take a
<trigger-id>, but no command listed them. The ids were reachable — the
detail payload returns "triggers" as a top-level sibling of "autopilot",
and the prefix resolver already reads it — yet that requires knowing the
envelope shape, so in practice the reported workaround was the web UI.
The prefix resolver's own miss message already told users to "run the
list command", which did not exist.

Add `autopilot trigger-list <autopilot-id>`, reusing the detail payload
and the same webhook-credential redaction `get` applies, so ids, kind,
enabled state and schedule are readable without --output json | jq.

Also add NEXT_RUN to the `autopilot list` table. next_run_at has been in
the list payload all along, but the table dropped it, which made a
correctly scheduled autopilot indistinguishable from one with no trigger
at all — the second misdiagnosis in the same report. Times render
relative ("in 2h", "3d ago", "—") to stay narrow and to make "never
scheduled" a visibly distinct value rather than an empty cell; LAST_RUN
moves to the same format for consistency within that table. The `get`
table keeps absolute timestamps — it is a single-row detail view where
precision matters more than width.

Reported at https://github.com/multica-ai/multica/issues/7549

Co-authored-by: multica-agent <[email protected]>

* docs(cli): correct autopilot trigger documentation (MUL-6680)

The trigger section claimed webhook and api kinds were "not surfaced
here" because "there is no server endpoint that fires them yet". Webhook
triggers have both an endpoint and CLI support (`trigger-add --kind
webhook`, `trigger-rotate-url`); only the api kind is unsurfaced.

Also state that `triggers` is a top-level key alongside `autopilot`
rather than nested inside it — reading `.autopilot` and finding no
triggers is what the report described — and document trigger-list.

Co-authored-by: multica-agent <[email protected]>

* fix(autopilot): fail closed when the subscriber read fails (MUL-6680)

Both subscriber projections swallowed a query error and continued with an
empty slice. Because subscribers has no omitempty and is documented as
authoritative, that degradation serialized as "subscribers": [] on a 200
— recreating the exact defect this branch set out to remove: the caller
cannot distinguish "none configured" from "read failed", and a client
that round-trips the response into a full-replace PATCH would silently
wipe a real subscriber list.

An empty array is a claim, not an absence. Return 500 on both the list
and detail paths so the failure is visible and retryable.

The detail path carried this degradation before this branch; it is fixed
here rather than deferred because the list/detail agreement asserted by
TestListAutopilots_SubscribersMatchDetail would otherwise still break
under a failed read — leaving the field untrustworthy in exactly the
condition the fix is about.

Addresses review feedback on #7552.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-26 01:28:35 +08:00
Yuri GuiandAgent Op f8ec870f30 MUL-6629: feat(agents): add per-agent starter prompts for chat (#7261)
* feat(agents): add per-agent starter prompts

Co-authored-by: Agent Op <[email protected]>

* fix(agents): default omitted starter prompts

Co-authored-by: Agent Op <[email protected]>

* fix(migrations): renumber agent starter prompts

Co-authored-by: Agent Op <[email protected]>

* fix(chat): make starter prompt prefills reliable

Co-authored-by: Agent Op <[email protected]>

* fix(agents): preserve starter prompt writes

Keep dirty starter-prompt edits across agent refetches and fail closed when an older server cannot persist them. Clarify that copied starter prompts have no dedicated CLI override.

Co-authored-by: Agent Op <[email protected]>

* fix(migrations): avoid starter prompt prefix collision

Main now uses migration prefixes through 403, so move the starter-prompts migration to the next available prefix before requesting another review.

Co-authored-by: Agent Op <[email protected]>

* fix(agents): preserve edits after failed save

Ignore optimistic instruction snapshots while a save is pending so a rejected update can roll back the cache without erasing the user's submitted starter prompts. Cover the delayed optimistic-update and rollback sequence for both label and prompt fields.

Co-authored-by: Agent Op <[email protected]>

---------

Co-authored-by: Agent Op <[email protected]>
2026-08-25 22:43:16 +08:00
YYClaw 9e2d9b6d1c MUL-6661 feat(channels): add /new and /clear conversation controls (#7468)
* feat(channels): add new and clear conversation controls

* fix(dingtalk): resolve bot identity before dispatch

* feat(chat): expose channel-created chats consistently

* refactor(channels): snapshot task delivery routes
2026-08-25 18:28:01 +08:00
46b5d9e6d1 MUL-6658 fix(agent): own the runtime process tree on every backend (#7522) (#7531)
* fix(agent): own the runtime process tree on every backend

Cancelling a task on Windows left the agent running. The daemon marked
the task cancelled, revoked its mat_ token, and killed what it thought
was the CLI — but on Windows the direct child is usually a cmd.exe shim
and the real CLI is already a grandchild, so it survived. One reporter's
orphan kept working for 40 minutes, read the daemon owner's profile PAT
after its own token started returning 401, and posted comments as the
member (GH #7522).

Whole-tree ownership already existed on both platforms. It was opt-in,
and opt-in rotted: 8 of the 27 places this package starts a process asked
for a Unix process group, and 3 of the 23 backends took Windows Job
Object ownership. Adding claude to that list would have left 19 backends
with the same hole, so the fix is structural instead.

- newRuntimeCmd, at the single point where a runtime process is built,
  now puts every command in its own process group and replaces os/exec's
  leader-only Cancel with a group-wide kill. A backend that wants a
  graceful shutdown still assigns its own cmd.Cancel and wins. This also
  covers the --version and model-discovery probes, which cmd.Output()
  runs without ever reaching a start helper.
- startOwnedProcessTree is now the only way this package starts a
  process, so Windows Job Object ownership is not something a backend
  can forget. Each start is paired with releaseProcessGroup after the
  reap, which drops the job handle and kills whatever outlived it.
- Two structural tests enforce both halves, the way
  TestOnlyLaunchGoSpawnsRuntimeProcesses already enforces the argv
  boundary: a new backend calling cmd.Start fails, and one that takes
  ownership without releasing it fails.

The Unix half is #5918 for the 20 backends the original fix did not
reach: their CLI was never in its own group at all, so no group-wide
signal could have reached it.

Co-authored-by: multica-agent <[email protected]>

* fix(agent): own the process tree for synchronous probes too

Review found the ownership boundary had a hole the PR claimed to cover.
os/exec's Run, Output and CombinedOutput call Start themselves, so a
probe written with cmd.Output() never reaches startOwnedProcessTree and
owns nothing on Windows — where the direct child of a `--version` probe
is typically the npm shim and the real CLI is already a grandchild. That
is the same escape #7522 was reported for, in 15 call sites nobody was
looking at; detectCLIVersion's own comment already describes a broken
CLI leaving grandchildren that hold the stdout pipe.

- runOwned / outputOwned / combinedOutputOwned give a synchronous probe
  the same start, reap and release a task launch gets. They keep the
  stdlib contract: stdout returned, stderr attached to the ExitError,
  combined output through a single writer so os/exec gives it one pipe.
  The stderr sample is bounded, as os/exec bounds its own.
- All 15 probe sites route through them, and the structural guard now
  rejects Run/Output/CombinedOutput alongside Start.

The release guard did not hold its invariant either: it only asked
whether a file contained any releaseProcessGroup, so a second unreleased
start in a file that already had a pair would pass — and models.go is
exactly that shape. It now works per function: one owned start each,
paired with at least one release. Equal counts would be the wrong rule,
since dsh's Execute legitimately releases on two exit paths.

Both guards are now pure functions over an AST, with tests that feed
them synthetic sources — including the second-unreleased-start case that
the file-level version missed — so the guards fail closed without
needing a real regression committed to find out.

Adds the Windows probe regression (descendant must be in the job and die
with a cancelled probe) plus unix equivalents, and names the consequence
in the degraded-ownership warning, which is the only signal an operator
gets that cancellation on that host is back to killing the leader alone.

Co-authored-by: multica-agent <[email protected]>

* fix(agent): bound a finished probe's wait on pipes its descendants hold

Review caught that runOwned could wait for its own cleanup and never
return, and it reproduces: a descendant that inherited the output pipes
holds them open after the leader exits, cmd.Wait blocks until the copy
goroutines see EOF, and what would close those pipes is the release that
runs after Wait.

Cancellation breaks the cycle, which is why every test in the last round
missed this — they all cancelled first, and they all set WaitDelay by
hand. Normal exit has nothing to break it, and checkOpenclawVersion runs
on the caller's context before runContext establishes any task timeout,
so a version wrapper that exits cleanly while leaving a pipe-holding
descendant would hang the task until a human cancelled it.

runOwned now fills in a bounded WaitDelay when the caller left it zero —
Go defines WaitDelay for precisely "a child process that exits but
leaves its I/O pipes unclosed" — and signals the group before dropping
ownership, so the descendant is reaped on Unix too, where releasing a
process group is a no-op. A caller with its own bound keeps it;
detectCLIVersion has set one by hand since MUL-3812 for this same shape.

Adds the normal-exit regression on both platforms: the leader exits
holding nothing, the descendant keeps the pipes, and outputOwned has to
return in bounded time with the descendant gone. The Windows descendant
spawner gains a spawn-and-exit mode and now hands its stdio to the
descendant, which is what makes it hold the pipes at all.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-25 16:27:46 +08:00
da7131ab59 MUL-6625: enforce prepaid member seat capacity (#7491)
* MUL-6625: enforce prepaid member seat capacity

Co-authored-by: multica-agent <[email protected]>

* fix(seat-capacity): address review findings

Co-authored-by: multica-agent <[email protected]>

* fix(seat-capacity): close retry and worker races

Co-authored-by: multica-agent <[email protected]>

* fix(seat-capacity): bound lock waits during rollback

Co-authored-by: multica-agent <[email protected]>

* fix(seat-capacity): pin the cross-repo confirm settlement budget

Multica Cloud must wait longer than this worker's whole retry budget before it
treats a member row as one its ledger never recorded; adopting a member whose
confirm is still retrying counts one person as two seats, and an over-estimate
is what asks a customer to buy seats they do not owe.

That budget was only implicit here — 10 attempts of exponential backoff plus a
tick and a lock wait each — and Cloud's horizon had been set from a single
retry. Extract the backoff into retryBackoff, expose SettlementBudget, and
assert the worst case (29m15s) stays inside Cloud's 45m horizon, so raising
maxAttempts or the backoff cap fails review here instead of widening the window
in which Cloud cannot quantify a ledger.

Co-authored-by: multica-agent <[email protected]>

* Revert "fix(seat-capacity): pin the cross-repo confirm settlement budget"

This reverts commit d0ec36f647.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Sol-Boy <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-25 12:51:44 +08:00
750fc0cc5a MUL-6655 Fix MCode ACP session startup race (ZIC-199) (#7516)
* Fix MCode ACP session startup race

Co-authored-by: multica-agent <[email protected]>

* fix(agent): only trust transport signals for MCode startup exit

mcodeSessionStartupExited also matched "process exited" / "broken pipe"
anywhere in the error text. A still-running MCode reports session/new
failures as a JSON-RPC *acpRPCError, and session/new is the call that
launches mcpServers, so a real error such as `failed to start MCP
server "filesystem": process exited with code 1` was rewritten into
"MiniMax Code ACP exited before starting a session; ... upgrade
MiniMax Code". That branch returns a fixed string with no %v, so the
original error was dropped, not just relabelled, pointing the member
at the wrong subsystem.

Drop the text matching and keep the transport-level signals. Real
provider exits already arrive as errMcodeProcessExited via
closeAllPending, and a stdin write racing the exit arrives wrapped as
syscall.EPIPE, so no covered case is lost.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: multica-agent <[email protected]>
Co-authored-by: J <[email protected]>
2026-08-25 12:30:52 +08:00
YYClaw 8c563b49be MUL-6555 feat(issues): preserve source context for sub-issues (#7461)
* feat(issues): preserve source context for sub-issues

* fix(issues): address source context review findings

* fix(migrations): guard source context rollback preflight
2026-08-25 12:00:49 +08:00
2aa45853d9 MUL-6640: fix stale Autopilot subscribers after member removal (#7501)
* fix(autopilot): prune departed subscribers

Co-authored-by: multica-agent <[email protected]>

* fix(autopilot): serialize subscriber saves with revocation

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Sol-Boy <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-24 18:49:54 +08:00
b2b4699ff6 MUL-6632: add status and priority filters to Inbox (#7497)
* feat(inbox): add status and priority filters

Co-authored-by: multica-agent <[email protected]>

* fix(inbox): degrade filters for legacy backends

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-24 16:27:42 +08:00
bdcb231ba0 MUL-5466 fix(agent): deliver the qwen prompt on stdin, not argv (#7343)
* fix(agent): route qwen launch through the Windows invocation chooser

On Windows npm installs qwen as qwen.cmd, a batch shim. Spawning a .cmd
goes through cmd.exe, whose %* forwarding re-tokenises the raw command
line, so the `-p <prompt>` value and the `--output-format stream-json`
flags buildQwenArgs puts behind it are mangled whenever the daemon's
structured chat prompt carries newlines, quotes or `&`. Qwen then runs
without a usable prompt or without stream-json, exits 0 after emitting a
single unparseable stdout line, and finalizeStreamResult reports "qwen
stream ended without terminal result". A short manual `qwen -p "hallo"`
never hits the mangling, which is why the CLI looked healthy in isolation.

Mirror the existing cursor/pi/copilot fix instead of inventing a new
mechanism: resolve qwen.ps1 next to the .cmd and invoke PowerShell with
`-File <ps1>` directly so Go hands each argv token whole, staying on the
official launch path. qwenBackend.Execute now builds its process through
Command.execVia(chooseQwenInvocation, ...) rather than Command.exec; on
macOS/Linux the chooser is a passthrough and behaviour is unchanged.

Fixes #6082

* test(agent): cover qwen's stdin prompt delivery and update signatures

buildQwenArgs dropped its prompt parameter in the prior commit (the
prompt now goes on stdin, not argv), so the existing unit tests needed
their call sites and assertions updated to match — including an
explicit check that "-p" never appears in argv again.

Adds TestQwenBackendDeliversPromptOnStdin: a round-trip regression
guard proving a prompt containing embedded double quotes, an em dash,
and shell metacharacters reaches the child process byte-for-byte via
stdin, the way cursor-agent and Pi already do (#5649's fix). The fake
qwen script now optionally captures stdin to a file the same way it
already captures argv.

* test(agent): cover qwen Windows PowerShell launch

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-24 15:08:14 +08:00