4 Commits
Author SHA1 Message Date
13eb4bda93 MUL-8154 fix(daemon): enforce runtime ownership on daemon routes (#9423)
* fix(daemon): enforce runtime ownership on daemon routes (MUL-8154)

/api/daemon/* accepts any workspace member's PAT/JWT, and authorization only
checked workspace membership. Any member could claim another member's tasks
(receiving the agent's custom_env, MCP config and an owner-scoped task token),
write task results/messages, and take a runtime over by re-registering its
daemon_id.

- Runtime-level check on every runtime/task daemon route, HTTP heartbeat,
  deregister, batch claim (and so the WS claim RPC) and the WS handshake:
  user credentials must own the runtime (NULL owner fails closed); daemon
  tokens must match a non-empty runtime daemon_id.
- Register: pre-check that every existing row on the daemon_id belongs to the
  caller, plus a required_owner_id fence in both upserts (incl. the
  failed-profile path) so a concurrent registration cannot win the row.
  Daemon-token registration keeps the existing owner.
- Legacy merge: only for user-credential registrations, and only when the
  caller owns both rows, verified on the FOR UPDATE-locked rows.
- Tasks without a runtime are not writable through daemon routes; status
  stays readable.
- GetTaskStatus / WS lease queries carry runtime owner/daemon so the hot
  paths keep one round trip.

Co-authored-by: multica-agent <[email protected]>

* test(daemon): authenticate fixtures as the runtime owner (MUL-8154)

Most handler tests drove daemon routes with a daemon token whose daemon_id
did not match the fixture runtime, relying on the workspace-wide token scope
this fix removes. They now authenticate the way real daemons do, with the
runtime owner's user credential (newOwnerDaemonRequest). Tests about daemon
token scoping keep mdt_ and get fixtures with a matching daemon_id; the
ownerless-runtime tests also assert that a user credential is refused.

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): serialize registration per daemon and keep Qoder WS claims working (MUL-8154)

Review of #9423:

- P1: the owner pre-check and the per-provider/profile upserts hit different
  conflict keys, so two members registering different providers of the same
  daemon could both pass the pre-check and both insert, leaving one daemon
  with two owners (and a daemon token for either runtime reaching the
  other's). Registration now runs the pre-check and every runtime write in one
  transaction under a pg_advisory_xact_lock on (workspace_id, daemon_id), on a
  single connection so queued registrations cannot starve the holder. Legacy
  merges, analytics and the register event run after commit; failed-profile
  records use savepoints so they stay best-effort.
- P2: the Qoder revocation check read the Authorization header, which the WS
  RPC bridge's in-process request does not carry, so valid Qoder WS claims
  silently returned nothing. DaemonAuth now records the mdt_ token hash in the
  request context, the WS identity keeps it, the RPC bridge re-attaches it,
  and the check validates that hash on every call (revocation still applies
  to an open connection).
- Regression tests for cross-provider and provider+profile concurrent
  registration, and for valid/revoked Qoder credentials over WS RPC.

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): bind daemon tokens to the runtime owner they were issued for (MUL-8154)

Second review of #9423: a daemon_id can be re-registered by another member
after its runtimes are deleted, and a daemon token was bound only to
(workspace, daemon_id), so the previous owner's Remote MCP token reached the
new owner's runtime (heartbeat, claim, owner-scoped task token).

- daemon_token.owner_id (migration 588). Remote MCP tokens are bound to the
  locked runtime owner inside FinalizeTaskClaim; live tokens are backfilled
  only where their daemon has exactly one owner.
- DaemonAuth carries the owner binding (DB and cache; cache key bumped to v2
  so pre-upgrade entries are never read), the WS identity keeps it and the
  RPC bridge replays it.
- Daemon-token authorization now also requires token owner == runtime owner.
  Qoder-managed tokens stay unbound (governed by cloud_agent_connection), and
  user credentials can no longer register the reserved qoder-managed-* ids.
- Tests: the reuse lifecycle over HTTP, a new WS handshake and an already
  open WS connection; the reserved daemon_id; the concurrency barrier now
  hooks the registration transaction (TxStarter) so the interleaving is real.

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): owner-bind daemon-token registration; drop unsafe token backfill (MUL-8154)

Third review of #9423:

- Migration 588 no longer backfills daemon_token.owner_id from the daemon's
  current owner: that owner may be a later re-registrant, which re-bound the
  previous owner's live token to them. Existing non-Qoder tokens stay NULL and
  are refused (24h Remote MCP credentials, re-issued on the next claim).
- owner_id is a plain UUID column, no foreign key / cascade (AGENTS.md).
- Register via a non-Qoder daemon token now uses the token's owner for the
  in-transaction pre-check and the SQL owner fence, and refuses unbound
  tokens. Qoder keeps its connection-based check; daemon tokens still never
  trigger legacy merges.
- Tests: an old owner's bound token and an unbound legacy token can neither
  register over nor heartbeat the replacement owner's runtime.

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): grant the Qoder registration exemption only with a live connection (MUL-8154)

Fourth review of #9423: register skipped the token owner binding for any
qoder-managed-* daemon token, but checked the Qoder connection only when the
body declared a qoder_cloud runtime. A pre-upgrade Remote MCP token under that
prefix (no owner, no connection) could register type=claude and rewrite
another member's runtime.

Register now verifies the token against a live cloud_agent_connection before
anything else whenever the token is in the Qoder namespace, regardless of the
declared runtimes, and restricts that namespace to the bridge's own shape: a
built-in qoder_cloud runtime, no custom profiles, no failed_profiles.

Test covers built-in, custom-profile, failed-profiles-only and qoder_cloud
bodies with an unconnected token, plus the connected token's allowed and
refused shapes.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: multica-eve <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-10-22 16:47:06 +08:00
78a5f69ae1 MUL-6813: fix: surface private runtime owner mismatch as explicit failure (#7805)
* fix: fail private runtime owner mismatches (PUCK-89)

Co-authored-by: multica-agent <[email protected]>

* fix: keep runtime mismatch repair off idle claim path

* test: cover mixed runtime claim outcomes (PUCK-89)

Co-authored-by: multica-agent <[email protected]>

* fix: settle runtime owner mismatches during claim

* fix: reject ownerless agents on private runtime claims

* chore: restore claim version sampling layout (PUCK-89)

Co-authored-by: multica-agent <[email protected]>

* fix: re-authorize delivery at the finalize boundary, dedicated runtime_access_denied client copy, fixture convention (PUCK-89)

Blocker 1 — final delivery gate: FinalizeTaskClaim now runs a caller-
supplied authorize closure inside its transaction. The singular and
batch claim paths share finalizeClaimDelivery, which re-locks the
runtime row (FOR UPDATE), re-reads the agent, and re-verifies the
private-runtime owner fence against CURRENT ownership before the task
token commits. A concurrent re-registration that would change
owner_id blocks until the gate commits, closing the stale-snapshot
TOCTOU. Mismatched tasks settle through the existing
failClaimedTaskBeforeLaunch -> FailTask path; singular keeps
200 {"task":null}, batch skips the task and keeps returning valid
ones. Regressions cover both paths.

Blocker 2 — client copy: runtime_access_denied gets dedicated
actionable copy (make runtime public / rebind-copy agent) on the
issues blocked-trigger mapping, both chat send-failure toasts, the
chat failure-reason map, the mobile dispatch-reason helper, and the
autopilot run-now toast. Generic fallbacks unchanged; locale keys
added for en/ko/ja/zh-Hans.

Blocker 3 — fixture convention: runtime_access_denied_test.go now
uses testutil.Call; daemon_runtime_access_test.go seeds its chat task
via the dbfx.Task fixture. Test semantics unchanged.

Co-authored-by: multica-agent <[email protected]>

* fix: bind final task token to locked runtime owner

Co-authored-by: multica-agent <[email protected]>

* fix: keep delivered user context on current owner

Co-authored-by: multica-agent <[email protected]>

* test: use testutil.Call in settlement-failure claim regression

Convert the last manual httptest.NewRecorder flow in
TestFinalizeClaimDelivery_SettlementFailureIsUnsettled to the repository's
required testutil.Call helper (hard convention from review) and drop the
now-unused net/http/httptest import. No production code changes.

Co-authored-by: multica-agent <[email protected]>

* PUCK-132: settle queued private-runtime owner mismatches as runtime_access_denied

Align persisted task failure semantics with admission semantics: queued
private-runtime ownership mismatches now settle as runtime_access_denied,
letting clients reach the dedicated recovery copy instead of the generic
invalid_task_identity copy. Actual task/agent identity violations
(agent rebound, agent deleted, response identity mismatch,
error_agent_runtime_changed) keep invalid_task_identity.

- add taskfailure.ReasonRuntimeAccessDenied (permanent, non-retryable
  ownership authorization failure; agent process never launched)
- swap the reason in the two owner-mismatch settlement paths in
  handler/daemon.go (response-assembly recheck + delivery-gate
  authz-error default branch, including ownerless-runtime denial)
- regression A (queued issue task), B (queued chat task + assistant
  failure message via existing FailTask path), C (agent rebind keeps
  invalid_task_identity)

Co-authored-by: multica-agent <[email protected]>

* PUCK-132: map persisted runtime access failures in clients

* fix(i18n): add missing fr translations for runtime_access_denied keys

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: multica-agent <[email protected]>
Co-authored-by: worker-opencode <[email protected]>
Co-authored-by: puck-181 <puck-181@local>
2026-09-18 15:39:25 +08:00
f39debf8d1 MUL-7344: skip the issue read and the comment scan when the server already answered them (#8377)
* feat(daemon): read the comment delta with one --since, not a roots scan (MUL-7344)

A resumed follow-up turn whose claim carried a server-computed issue-wide
comment delta was still told to run the workflow's two-step scan: a
`--roots-only --summary` pass to discover which threads moved, then a
`--thread ... --since ...` expansion per thread. That rediscovers, in two or
more calls, an answer the server already has.

`--since <anchor>` without `--thread` IS that answer.
ListCommentsSinceForIssue returns every comment on the issue created after the
anchor, in every thread and at every nesting level, with no folding, on the
same `created_at > since` predicate CountNewCommentsSince counted with. It is a
strict superset of what the roots scan could surface: roots-only
`last_activity_at` is MAX(created_at) over a thread, so any comment able to
move a thread's activity is already in the `--since` result.

So BuildNewCommentsHint now hands over exactly one wide read and says it
answers step 2's scan — the same rule BuildResumedCommentsHint already applies
to a server-computed EMPTY delta. The optional second command stays a THREAD
read on `--tail 30` for the reply itself; it is never the wide read, and it
never combines `--thread` with `--since`, which drops the thread root.

Brief step 2 and the multica-platform skill's bounded-reads rule both gain the
matching static sentence, so an agent is not left choosing between the hint and
two surfaces that contradict it.

Cold start, dropped resume and unknown-delta paths are untouched, byte for
byte: only an affirmative server report may stand in for the scan.

Co-authored-by: multica-agent <[email protected]>

* feat(claim): compare the issue against the last run so a resumed turn can skip the read (MUL-7344)

A follow-up turn that resumes its provider session already holds the issue it
read minutes ago, and was still told, unconditionally, to run `multica issue
get` before doing anything. On an issue that did not move, that read returns
bytes the session already has.

The claim now answers the question instead. Each claim records the comparable
state of the issue it built its payload from — status, assignee, priority, and
sha256 of title and description — in a new `agent_task_queue.issue_snapshot`
JSONB column, and compares against the snapshot the SAME agent's previous run
on the SAME issue recorded. The response carries `issue_state_delta_known`,
`issue_changed_fields`, and the current `issue_status` / assignee.

Why a per-claim snapshot and not a `content_revision` on `issue`: a revision
column has to be bumped by every writer that touches one of those five fields,
and the first one missed reports "unchanged" forever with no symptom. A
snapshot taken at claim time has exactly one writer. It also rides the anchor
the comment delta already uses (agent_id, issue_id, started_at DESC), so both
deltas on a claim mean "since your last run" in the same sense.

The daemon renders one of three things, gated on a REAL resume:

  - unchanged  → reported as workflow step 1's answer; the read survives only
                 as "if resumed memory is not enough".
  - changed    → the changed field NAMES plus the read, so the agent knows what
                 to look for instead of diffing blind.
  - otherwise  → the pre-existing instruction, byte for byte.

`issue_state_delta_known` carries the same contract as
`new_comments_delta_known` (MUL-6984), and for the same reason: a cold start,
a missing prior snapshot, a read error, a foreign shape version and an old
server all produce an empty changed-field list, identical to a genuine
"nothing changed". Only the flag separates them, and only the affirmative one
may stand in for the read.

The compared set is named in full in the prompt rather than summarised as "the
issue": labels, parent, due date, stage, project and metadata are NOT compared,
and an agent told "unchanged" has to be able to see that.

The snapshot write is NOT gated on the task being comment-backed — an
assignment run that recorded nothing would leave the next run with no baseline.
It shares the receipt's CAS, so a stale handler cannot write over a newer
reclaim or after execution starts, and a snapshot that fails to encode is
simply not recorded: the next run reports "not compared" and reads the issue.

Co-authored-by: multica-agent <[email protected]>

* fix(claim): renumber the migration past main and fold in review nits (MUL-7344)

CI caught a real collision: main landed 471_comment_deleted_at through
474 while this branch was open, so 471_agent_task_issue_snapshot shared a
numeric prefix and TestMigrationNumericPrefixesAreUnique failed. Renumbered to
475 after merging main; no other migration references it.

Review nits from the PR review, all non-blocking, all cheap to take now:

1. Brief step 2's closing sentence told the agent that a resumed run's scan
   surfaces moved threads via `last_activity_at`. That is now true only on the
   path where the delta is UNKNOWN — when the server hands over a `--since`
   delta there is no scan to read it from. Rewritten to attach the
   `last_activity_at` guidance to the scan itself, wherever the scan happens.
   Done in this PR rather than deferred because this PR already moves brief
   bytes, so it costs no extra prompt-cache invalidation.

2. The two per-claim anchor reads are now one. GetLastTaskStartedAtForIssueAndAgent
   and GetLastTaskIssueSnapshotForIssueAndAgent selected the same row with the
   same WHERE and ORDER, differing only in projection; they are replaced by
   GetLastRunAnchorForIssueAndAgent returning started_at and issue_snapshot
   together. A comment-triggered claim is back to ONE anchor read, as before
   this branch.

   The saved round trip is the smaller half. The real gain is that "since your
   last run" is now a single resolved fact on a claim instead of two
   independently-resolved ones that could disagree. The shared read also shares
   a failure mode, and both consumers already degrade the same safe way: comment
   scan required, issue comparison unknown.

3. The hint's `%d new comment(s)` excludes the injected trigger and the agent's
   own comments; the `--since` read it hands over has no such filter, so the row
   count is legitimately higher. Narrowing the read to match would mean a second
   count-shaped query for no gain, so the text now states the mismatch and its
   direction outright instead of leaving the agent to reconcile two numbers.

New test pins the merge: one claim, one prior run, both deltas resolved from
that single anchor row.

Co-authored-by: multica-agent <[email protected]>

* fix(migrations): renumber the issue-snapshot migration to 478 (MUL-7344)

main landed 475_issue_status_key_not_reserved through 477 while the previous
renumber was in flight, so 475_agent_task_issue_snapshot collided in turn.
Merged main again and moved to 478. TestMigrationNumericPrefixesAreUnique
passes, and the full ledger applies cleanly from scratch through 478.

Co-authored-by: multica-agent <[email protected]>

* fix(claim): measure both deltas from the run whose session is resumed (MUL-7344)

Review found a real correctness bug, not a performance one: the claim could tell
a resumed agent "the issue is unchanged" while the session it handed back
predated the change.

Both deltas were anchored on the most recently STARTED run for (agent, issue).
The session the daemon resumes is chosen by a different rule, and the two are
not the same row:

  - GetLastTaskSession skips poisoned and retired sessions, so a newer run that
    died on an unprocessable conversation is passed over for an OLDER healthy
    one.
  - A manual rerun resumes the exact source task the operator clicked.

So: an old session remembers description A, the user edits it to B, a newer run
records B and then dies unresumable. The next follow-up resumes the OLD session
but compares B against B, reports "unchanged", and the agent keeps working from
a context that still says A. PriorSessionResumeUnavailable does not catch this —
it flags a missing Codex rollout, which is only one of the ways the resumed
session can be older than the newest run.

The fix binds both deltas to the run actually being resumed. GetLastTaskSession
now returns that row's started_at and issue_snapshot alongside its session, the
rerun path takes them from its source task, and the delta computation moves
after resume resolution so it can only ever read the adopted row. Where no
session is adopted — no prior session, a session on another runtime, a
forced-fresh retry, a rerun whose source failure poisoned the conversation — no
delta is reported and the daemon performs the reads it always has.

That makes the standalone anchor query unnecessary: the anchor is now a
projection on a row the claim already fetched, so a claim reads no extra row at
all (GetLastRunAnchorForIssueAndAgent is removed).

The precondition is now visible in the fixtures: a prior run has to be
resumable for a delta to exist, so the comment-delta tests seed a session_id.
The regression test covers both paths and was verified to fail against the
previous anchor — both sub-cases report an empty changed-field list, which is
exactly the wrong "unchanged".

Also renumbers the migration 478 → 491; main landed 478_issue_status_category_expand
through 490 while the branch was open.

Co-authored-by: multica-agent <[email protected]>

* fix(claim): a never-started anchor row may date neither delta (MUL-7344)

Final review found the last corner of the resumed-run bug. Binding the anchor to
the resumed SESSION was necessary but not sufficient: GetLastTaskSession returns
that session's latest TERMINAL row, and that row is not always a run that ran.

A retry child can inherit the session, be claimed — which writes its snapshot —
and then fail before the agent ever starts: runtime offline during prepare,
cancelled while dispatched. Its snapshot describes the issue as of ITS claim,
while the session's memory stops at the last run that actually executed. If the
description was edited in between, the current issue matches that snapshot and
the claim reports "unchanged" to an agent whose context predates the edit.

The started_at gate now covers both deltas rather than only the comment one.
The comment delta was already immune because it read StartedAt directly; the
issue delta read only the snapshot, so it had no such guard. Hoisting the check
makes the row's requirement explicit — a row that never ran dates nothing — and
the inner duplicate is dropped so no reader has to wonder what each guards.

Regression test reproduces the exact fixture and was verified to fail without
the gate: known=true with an empty changed-field list, i.e. the false
"unchanged".

Also from review:

- Adopt the no-adopted-session table (missing session, session on another
  runtime, force-fresh retry, poisoned rerun source). Each seeds a usable
  snapshot AND an unread comment, so a regression has something to report and
  the assertion cannot pass on an empty fixture.
- The resumed-run regression test asserted a comment anchor that never
  executed: both sub-cases created only the trigger comment, which the count
  excludes, so NewCommentsSince was empty and the branch was dead. A comment
  between the two runs now makes it run and pins known/count/since.
- Rewrite the migration header, which still described the old
  "(agent_id, issue_id, started_at DESC)" anchor and an index rationale for a
  column that is never a lookup key.

Co-authored-by: multica-agent <[email protected]>

* refactor(claim): compare title, description and status only (MUL-7344)

The snapshot compared five fields. The comparison answers exactly one question
— must the agent run `issue get` again? — so a field earns a place only if
changing it alters what the agent does AND its current value is not already in
the per-turn message.

Assignee fails the second test: the claim ships the current assignee outright,
and "is this mine now" is answered by that value, not by whether it moved.
Priority fails the first: it is reachable only by reading the issue, but it does
not change what the agent does. Both are dropped.

Status stays even though it is also shipped as a current value, so comparing it
is strictly redundant for "what is it now". It earns its place as a signal
rather than a value: "changed: status" is the clearest evidence that somebody
intervened between the runs — a push back from in_review to todo means the
delivery was rejected — and it costs nothing to carry.

Narrowing the set means a narrower claim, so the prompt now says "compared
title, description and status". An issue whose only change is assignee or
priority is reported as unchanged, which is correct and now pinned by a test
that also asserts the current assignee still ships — without that second
assertion the fixture's UPDATE could silently not apply and the test would pass
on nothing.

Snapshot version goes to 2, because the rule is that the version tracks the
compared SET. No deployment has written a v1 row, but the gate is what makes
narrowing safe in general: a v1 row read by v2 code is "not compared", which
degrades to the unconditional read.

Test fixtures now build snapshots at issueSnapshotVersion rather than a literal
1, so a future bump cannot quietly turn them into not-compared tests while they
keep passing. The foreign-version case stays explicit at +1.

Also renumbers the migration 491 -> 495; main landed 491 through 494 while the
branch was open. Fifth renumber.

Co-authored-by: multica-agent <[email protected]>

* refactor(claim): keep the snapshot at v1 (MUL-7344)

The previous commit bumped issueSnapshotVersion to 2 when it narrowed the
compared set. That was the version rule applied past its purpose.

The rule — "bump when the compared SET changes" — exists to stop a stored
snapshot from being compared against a differently-shaped one. It protects
DEPLOYED rows. This column is introduced by this same unmerged PR: the
migration exists only on this branch, main has neither the column nor the
writer, and no environment has ever stored a row with the wider set. There is
nothing for the bump to protect, and shipping v2 as the first version the
column has ever held would leave a reader hunting for a v1 that never existed.

Back to 1. The `v` field and its unknown-version gate stay exactly as they
were — those are what make a real narrowing safe later. The comment now says
which version this is and why the review-time narrowing did not earn a bump,
so the rule still has a stated precedent without a burned number.

No behaviour change: the compared set is still title, description and status,
and fixtures build at issueSnapshotVersion rather than a literal, so they
followed the constant back down.

Co-authored-by: multica-agent <[email protected]>

* fix(claim): only a completed run may date a delta (MUL-7344)

Final review found a window one step later than the never-started case.

A snapshot is written at CLAIM time. The daemon then writes started_at
(TaskService.StartTask) and only afterwards launches the provider. So a run can
carry both a snapshot and a started_at while its prompt never reached the
session: the daemon exits during prepare, the runtime drops, orphan recovery
marks the row failed. Anchoring there compares the current issue against a
snapshot describing an edit that session never saw — reporting "unchanged"
across a real change, and dating the comment delta late enough to hide every
comment older than that row's start. Reproduced on the real
CreateRetryTask -> claim -> StartTask -> RecoverOrphanedTasksForRuntime ->
follow-up claim path.

Nothing on the row separates "failed after the provider ran" from "failed
before it ran". session_id is inherited by CreateRetryTask at insert, so it does
not mean the session was fed; started_at precedes the launch, so it does not
mean the turn began. The only terminal status that PROVES delivery is
'completed' — a run cannot finish a turn it never started.

So failed and cancelled sources now date neither delta. Their SESSION is still
resumed exactly as before, and GetLastTaskSession still accepts them on purpose;
those runs simply perform the context reads they always did. That is the cheap
side of the trade: a follow-up after a failure or a stop is already recovering
from something, while the case the optimization exists for — an ordinary
follow-up after a healthy run — is untouched.

Both conditions now live in newResumedRunAnchor rather than at the use site, so
"which run may date a delta" is answered once, and GetLastTaskSession carries
the status needed to answer it.

Regression test is the reviewer's repro, verified to fail without the rule:
known=true with an empty changed list AND a computed zero comment count, i.e.
both a false "unchanged" and a false "no new comments".

Also:
- Renumber the migration 495 -> 499; main landed 495 through 498. Sixth
  renumber.
- The migration's content_revision rationale named the old five-field set while
  arguing about hypothetical issue-row writers; it now refers to "any compared
  field", which is what the argument actually depends on.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-09-16 16:11:16 +08:00
bbba9c5460 MUL-6697: enforce runtime access on task claims (#7571)
* fix(tasks): enforce runtime access on claims

Co-authored-by: multica-agent <[email protected]>

* fix(tasks): settle invalid runtime claims explicitly

Co-authored-by: multica-agent <[email protected]>

* test(tasks): cover runtime owner claim mismatch

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Sol-Boy <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-26 15:55:05 +08:00