Commit Graph
7 Commits
Author SHA1 Message Date
8352787263 MUL-7658: fix(daemon): recover undelivered batch claims after timeout (ZIC-260) (#8774)
* fix(daemon): recover undelivered batch claims after request timeout

Co-authored-by: multica-agent <[email protected]>

* test(daemon): avoid copying task service locks in recovery fixture

Co-authored-by: multica-agent <[email protected]>

* test(tasks): isolate delegated recovery assertions (MUL-7658)

Co-authored-by: multica-agent <[email protected]>

* test(daemon): use one clock for deferred claim hints (MUL-7658)

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): keep claim recovery within client deadlines (MUL-7658)

Co-authored-by: multica-agent <[email protected]>

* fix(execenv): preserve racy Git index timestamps in snapshots (MUL-7658)

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): deliver claim responses before recovery side effects (MUL-7658)

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): isolate HTTP claim retries from recovery (MUL-7658)

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: multica-agent <[email protected]>
Co-authored-by: Sol-Boy <[email protected]>
2026-10-15 16:43:24 +08:00
6d5cdcbed2 MUL-7488: fix(execenv): exclude committed local HEAD changes from resume replay (#8549)
* fix(execenv): exclude clean HEAD advances from resume replay

* fix(execenv): normalize carried WIP before replaying local edits

Co-authored-by: multica-agent <[email protected]>

* fix(execenv): resume after conflicting local draft commits

Co-authored-by: multica-agent <[email protected]>

* fix(execenv): limit snapshot conflict handling to affected paths

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Sol-Boy <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-10-14 14:43:47 +08:00
61ea48fd2e MUL-6881 fix(daemon): keep one worktree branch per conversation in local worktree mode (#7815)
* fix(daemon): keep one worktree branch per conversation in local worktree mode

Every comment on a local_directory issue started a new task, and worktree
mode gave each one a branch of its own forked from the user's HEAD. Turn two
therefore stood in a tree that did not contain turn one's work, with that work
stranded on another branch and nothing saying so.

Tasks that share a conversation — the turns of one issue, one chat session —
now share a branch, agent/<agent>/<issue>, and a follow-up checks it out at its
tip instead of forking from HEAD. Env roots stay per task, so sibling tasks
still run concurrently; when one already holds the conversation's branch, the
other forks from its tip onto agent/<agent>/<issue>-<task> rather than falling
back to HEAD.

Replaying the user's uncommitted tree onto a continued branch cannot use the
turn-one path: that merge takes the user's HEAD as its base, so it conflicts
exactly when the agent edited the lines the user had left uncommitted. Each
turn now records the snapshot it saw at refs/multica/local-state/<branch> and
the next one replays only the difference — a cherry-pick whose parent is that
snapshot. A genuine conflict leaves the branch as it stands and logs which
files did not come across, rather than stranding the conversation.

Teardown follows the same rule everywhere: a branch this prepare did not
create belongs to the turns before it, so Finalize's read-only cleanup, Discard
and the prepare rollbacks only ever drop a branch they created themselves.

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): prove branch ownership and hand replay conflicts to the agent

Review of the per-conversation worktree branch found four ways it could put
the wrong tree in front of the agent. All four came from the same two gaps:
the user's snapshot did not describe their whole directory, and a branch was
recognised by its name.

The snapshot is now one commit covering tracked edits AND untracked files,
built in a private index inside the task's env root. It answers the question a
continued branch actually has to ask — what has the user changed since the
state I already carry — for every file, so a once-untracked file the agent
edited is no longer reverted to the user's older copy each turn, and their
later deletion of one carries. Redirecting GIT_INDEX_FILE also removes the
.git/index.lock race the previous capture had to retry around (#7434).

A conflict between the user's newer edits and the branch's own work is now the
agent's to resolve, per the product decision on the issue: the worktree is
handed over mid-merge, the turn's prompt names the files and says which side is
which, and Finalize refuses to deliver — preserving the worktree — while
anything is still unmerged. The snapshot only advances once the branch really
carries it, so an unresolved edit is offered again next turn instead of being
recorded as synced. Before this, the conflict was reset away and the user's
change was lost from every later turn.

Continuing a branch is decided by an ownership record — workspace, agent and
conversation ids, recorded in the snapshot commit at
refs/multica/local-state/<branch> — never by the branch name. A same-named
branch the user created is left untouched and the task takes
agent/<agent>/<issue>-<fingerprint>, which its own follow-ups then continue.
The same check keeps two workspaces that mint one issue identifier apart.

Those refs are now GC'd: every prepare drops the record of a branch that no
longer exists, and the docs say where they live and how to clear them.

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): tie a task branch's record to the branch, and quote untrusted paths

Ownership proved that Multica once wrote a branch of this name, which stayed
true after the user deleted it and created their own under the same name. No
prepare runs in that gap, so the orphan sweep never sees the branch missing and
the next turn continued straight into the user's unrelated work.

The record now also pins the branch tip it was written at, as the second parent
of the record commit, and a branch is only continued while that commit is still
an ancestor of its tip. Deleted-and-recreated, force-moved and rebased branches
all fail that test; committing on top of a delivered branch — the normal case —
passes it and picks the user's commit up. Finalize re-records afterwards,
because a checkpoint written at prepare time sits on the branch's base, which is
exactly where a recreated branch would sit.

Conflicted paths reaching the turn's prompt are quoted rather than wrapped in a
code span. They come from the user's repository and git allows newlines,
backticks and quotes in a filename, so a raw one could close its list item and
continue as an instruction line of its own; the same quoting is applied to the
Finalize error. The list is bounded at 50 names with the remainder counted.

Also drops a duplicated pair of bullets in the Chinese docs and corrects two
file comments that still described the stash-and-copy replay.

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): make the branch record part of delivery, and bound the conflict list by bytes

The record could still authorise a branch that was not ours. Finalize wrote it
after removing the worktree and treated a failure as non-fatal, so a failed
update-ref left the ref pointing at the record Prepare had written — whose
checkpoint is the branch's base, which is exactly where a branch the user
deletes and recreates from HEAD sits. The ancestor check then passed on their
work.

Two changes close it. A checkpoint is only written when it is a commit the
branch could not sit at without this conversation's work: Finalize records the
tip it actually delivered, and Prepare records early only once the branch has
moved off the user's HEAD — which is the baseline commit, or a continued
branch's own tip. And the record is now part of delivering: Finalize writes it
BEFORE removing the worktree and fails the task if it cannot, so the worktree is
still there to preserve, exactly as for a commit that could not be made. The
checkpoint is the commit the caller delivered rather than a re-read of the
branch ref, so a branch moved in that window fails the next ancestor check
instead of being recorded as current.

A sibling racing the very first turn of a conversation on a clean tree now finds
no record yet and starts its own line of work rather than forking from a branch
it cannot prove is ours. That is the price of the rule and it is the safe side.

The conflict list in the prompt is bounded by rendered bytes rather than by
entries — a git path is as long as the filesystem allows, so 50 entries bounded
nothing — with the remainder counted and a single oversized path unable to
overrun the budget either.

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): give every task branch a commit of its own, and verify what a run delivers

The checkpoint rule — record only a commit the branch could not sit at without
this conversation's work — held in Prepare and not in Finalize. Finalize took
whatever the worktree's HEAD was, so a run that reset its worktree back to the
user's own HEAD had that plain user commit recorded as the branch's checkpoint,
and a branch the user later recreated there passed the ownership check.

Two changes make the rule total. Every branch this mode creates now gets a
baseline commit of its own, empty when the user's tree was clean: without it the
branch sits exactly where the user's HEAD does and nothing tells it apart from a
branch they create at the same place. It also makes `git diff <baseline>..
<branch>` the agent's work on every branch rather than only on the ones that
started dirty, which is what the surrounding comments already claimed.

And Finalize verifies the delivery point before recording it: the tip has to BE
the task's branch — not a detached checkout, not a branch someone moved
underneath the run — and it has to still contain the anchor, which is that
baseline or the checkpoint the previous turn recorded and this one verified. A
delivery that fails either test is not recorded at all: the task fails, the
worktree is preserved, and the error names what is missing.

A sibling racing the first turn of a conversation forks from it again, since
every branch now carries a recordable checkpoint from the moment it is prepared.

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): keep the anchor on the commit that carries this turn's user edits

A follow-up turn commits the user's newest edits as its own baseline, but the
anchor stayed on the checkpoint the previous turn had recorded. A run that then
reset its worktree back to the previous tip still passed verification — the old
checkpoint is an ancestor of it — so the turn was recorded as having delivered
edits the branch no longer had. The turn after that computed its replay from a
state the branch never took, and the user's edits were simply absent from the
worktree with nothing to indicate it.

The anchor now follows every baseline this task commits, so it always names the
newest commit carrying the state about to be recorded. Resetting past it fails
the turn and preserves the worktree, as for the first-turn case.

The conflict path had the same hole from the other end: there the branch does
not carry the snapshot until the agent resolves and something is committed. When
a turn that started mid-merge ends with the branch exactly where it began, the
record keeps the state the branch is known to carry instead of advancing, and
the next turn offers those edits again. A resolution in favour of the branch's
own side leaves the same empty trace as discarding the merge, so this trades a
merge that may have to be concluded twice against recording edits as delivered
that may have been dropped.

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): measure a delivery against the commit its own turn started from

A turn that starts mid-merge only carries the user's edits once the agent
resolves and something is committed, and the guard for that compared the
delivered tip against the checkpoint an earlier turn had recorded. Those are not
the same commit: the user committing on a delivered branch is supported and
tested, so the branch tip a later turn starts from can already be past that
checkpoint. A resolution that committed nothing then left the tip equal to the
turn's own starting point but not to the older checkpoint, the guard did not
fire, and the edits were recorded as delivered — missing from the turn after
with nothing to indicate it.

Both questions now measure against LocalWorktree.BaseCommit, the commit this
turn actually started from. It is at least as new as the earlier checkpoint in
every case, so the separate anchor field is gone: a delivered tip that contains
this turn's starting point contains everything an earlier turn recorded, and a
tip that does not is a run that reset away work already delivered.

Co-authored-by: multica-agent <[email protected]>

* test(daemon): cover the claim-to-branch wiring with a two-turn Prepare test

Every multi-turn test so far drove PrepareLocalWorktree with hand-filled
params, which skips the step that turns a claim into them: which field becomes
the branch name, and which become the identity the branch is recorded under.
This one runs two turns of one issue through Prepare itself, and a third for
another workspace carrying the same issue identifier. Both halves of that
mapping are load-bearing — blanking the conversation key or fixing the
workspace id makes it fail.

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): carry the worktree's own state across the preparation helper

Prepare runs in a short-lived helper process and the Environment it built comes
back to the daemon as JSON. LocalWorktree keeps everything Finalize reasons
about in unexported fields, and ordinary struct marshalling drops those without
a word — so the daemon finalized a worktree whose owner, snapshot, tracksState
and createdBranch were all zero.

Nothing failed; the guarantees were simply absent in production. No delivery
verification ran, Finalize recorded nothing (leaving the checkpoint at the
baseline Prepare had written inside the helper), a conflicted turn could not
hold its snapshot back, and a read-only turn kept its branch. Every in-process
test passed throughout, because they call PrepareLocalWorktree directly.

Custom Marshal/Unmarshal now carry that state. aborted stays out of the wire
deliberately: the daemon sets it after Prepare returns, so it belongs to the
parent. The regression tests run two turns through PrepareIsolated — the real
boundary — and assert the state arrives, that Finalize's record names the tip it
delivered rather than the baseline, and that a read-only turn still drops its
branch.

Found by running the whole stack: a real daemon, a real local_directory in
worktree mode, and two real turns.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-09-01 13:50:52 +08:00
Bohan Jiang 0a54725fe6 MUL-6567 fix(execenv): lock a local_directory repo's git section across processes (#7462)
The lock meant to serialise git admin work on one repository was an in-process
sync.Mutex, and no two contenders were ever in one process: a daemon prepares
every task in a short-lived helper, and worktree mode deliberately skips the
per-path task mutex, so N concurrent tasks on one local_directory were N
processes issuing git commands at the same repo with nothing between them.

`git stash create`, which captures the user's uncommitted work before the
worktree is built, needs the repository's index lock and lost that race
routinely — reproducibly in about two runs out of three for two concurrent
prepares on a dirty repo. git reports the loss as exit status 1 with nothing
on stderr, and nothing retries a failed task, so the run died undiagnosable
and the issue rolled back to todo.

- lockGitRoot now takes an advisory file lock in the repository's COMMON git
  dir alongside the mutex, reusing the flock/LockFileEx primitives the env-root
  claim relies on. The common dir is what makes one lock cover one repository:
  the guarded section contends both for the per-worktree index.lock and for the
  refs / worktrees/ state every linked worktree shares. The file lock is taken
  before the mutex so waiting is not itself serialised. A git dir that cannot
  hold a lock file falls back to the mutex; a BUSY lock fails the caller.
- The capture retries, 5 attempts over ~1.5s, for the index-lock races owned by
  the user's own tools. `stash create` writes a commit object and returns its
  sha without touching the index, refs or working tree, so retrying repeats
  nothing.
- runGitStdout renders git's stderr into its error instead of collecting it
  into ExitError.Stderr and dropping it, and the capture names .git/index.lock
  when git itself says nothing.

Finalize and Discard take the same lock, so a prepare in a helper and a
finalize in the daemon parent no longer overlap. Finalize reports its worktree
as preserved if it cannot get the lock.

Seven tests, each verified to fail against the code it fixes, including a
second PROCESS holding the lock and two concurrent PrepareIsolated runs on one
repo. The timeout message is pinned never to advise deleting a held lock file:
the lock binds to an open file description, not a path, so unlinking it would
let two tasks into the same section at once.

Refs #7434
2026-08-24 01:12:39 +08:00
Bohan Jiangandmultica-agent 3622db21cf MUL-6491 fix(daemon): give each task an exclusively claimed env root (#7347)
* fix(daemon): derive task env roots and branches from the full task id

Task ids are UUIDv7, whose leading 8 hex chars are the high 32 bits of a
48-bit millisecond timestamp: they only advance once every 2^16 ms
(~65.5s). Deriving the env-root segment from that prefix meant every task
created inside one such window landed on the same directory.

Prepare removes an existing env root before recreating it, guarded only by
the comment "task IDs are unique". That premise held for the id but not for
the truncation, so the second task's Prepare ran os.RemoveAll over a live
sibling's env root — taking its workdir (including a worktree with
uncommitted work), output/, logs/ and the 0700 multica-config/ with it.
Overwritten CLAUDE.md / AGENTS.md identity files were the visible symptom;
the deletion was the actual damage.

Use the whole id for the env root and for the agent/<name>/<task> branch
names, which truncated the same way and collided for concurrent tasks of
one agent. shortID stays for logs and display only. Prepare now derives its
root through PredictRootDir so the two can no longer drift apart.

Existing short-prefix directories need no migration: the GC enumerates task
dirs with ReadDir and reads .gc_meta.json, so it still reclaims them.

Fixes #7326

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): take the task path segment from the id's random tail

The first attempt used the whole 32-char task id, which overflowed Windows
MAX_PATH: a branch name becomes a path under .git/refs/heads/ inside an
already-deep task checkout, and CI failed with

    cannot lock ref 'refs/heads/agent/windows-codex/<32 chars>':
    Filename too long

Length was never the fix. The bug was which end the segment came from: a
UUIDv7 leads with a 48-bit millisecond timestamp, so its first 8 hex chars
are the high 32 bits and only advance once every ~65.5s. Read from the tail
instead, where the id is random, and 12 chars are enough — 4 more than the
original segment rather than 24.

That trades a certainty for a probability, so make the destructive step
prove itself instead of assuming: Prepare stamps a .task_owner marker into
the env root and refuses to run os.RemoveAll over a root another task owns.
A rerun of the same task still resets its own directory. A task that fails
to start is recoverable; one that deletes a running sibling's uncommitted
work is not.

Two existing fixtures used synthetic ids differing only in the head and so
shared a tail. The guard caught them; they now differ where a real UUIDv7
does.

Fixes #7326

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): make the env-root claim atomic instead of check-then-delete

Review caught that the ownership guard did not cover the case it was added
for. It read .task_owner, then deleted, then wrote the marker — three steps,
so two tasks sharing a taskKey could both observe "no owner", and the slower
one would still os.RemoveAll the faster one's live directory. The guard only
held when a foreign marker was already durably in place, i.e. serially.

Replace it with a claim protocol built on two atomic filesystem primitives:
os.Mkdir fails if the directory exists, and O_EXCL fails if the marker
exists. Exactly one racing caller wins each, and a caller that wins neither
never reaches the destructive path.

The claim is then held for the rest of Prepare. A rerun clears the env
root's CONTENTS and leaves the marker in place rather than removing and
recreating the directory, so there is no window in which the root looks
unowned to a racing task.

A directory that holds files but names no owner now fails closed instead of
being deleted as abandoned. The marker is written before any content, so the
only unowned directory that is safe to adopt is an empty one — the residue
of a crash between Mkdir and the marker write, which would otherwise wedge
the task permanently.

Tests: a barrier-synchronised concurrent Prepare asserting exactly one task
starts and the winner's tree survives (fails against the previous sequence
with "directory not empty" / "invalid argument" — the deletion racing the
create), plus fail-closed and empty-adoption cases. repocache carried a
third copy of the task-segment rule with no test; it now has the same
contract test as the handler.

Fixes #7326

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: multica-agent <[email protected]>
2026-08-21 00:36:59 +08:00
Wu Changming 7fdc854c26 fix(issues): copy durable local path for worktree tasks (MUL-6378) (#7176)
A worktree-mode local_directory task runs in a disposable git worktree that
Finalize removes on every exit path, so the task's `work_dir` pointed at a
directory that no longer existed. "Copy local workdir path" handed the user a
path they could not `cd` into.

The daemon now reports a `durable_work_dir` handoff instead of having clients
infer one:

- `removeLocalWorktreeDir` returns an error and confirms removal with `Lstat`,
  so a broken symlink cannot be mistaken for an absent directory. Finalize
  fails when removal cannot be confirmed.
- `durable_work_dir` is written only after Finalize succeeds and the worktree
  is verifiably gone. A preserved worktree, an in-place run, a running task, a
  squad-leader task, and any older daemon all leave it unset, keeping the real
  `work_dir` authoritative.
- The field travels on the complete, fail, and cancel-ack callbacks and is
  persisted on the task row (migration 362, additive nullable column). Writes
  use COALESCE for replay idempotency; the cancel-ack path CASes on
  `status = 'cancelled'` so a late ack cannot touch a completed or failed row.
- The server derives `relative_durable_work_dir` through the same
  home-stripping helper as `relative_work_dir`; the absolute value is reserved
  for explicit clipboard actions and is never rendered.
- Clients select the durable path only for a terminal task that carries the
  field. The issue actions menu and the transcript dialog share one resolver,
  and the label and toast say the task worktree was reclaimed, naming the
  delivery branch when one exists.

The stored path is a point-in-time delivery snapshot: later renames or
detachments of the project resource do not rewrite historical tasks.
2026-08-20 01:30:58 +08:00
c9727e6b2b feat(daemon): worktree mode for local_directory resources (MUL-5707) (#6759)
* feat(daemon): worktree mode for local_directory resources (MUL-5707)

Local directory resources serialise every task on a per-path mutex, so two
tasks on one folder queue behind each other. This adds an opt-in worktree
mode: each task gets its own git worktree of the user's repo inside the
daemon's env root, so tasks run concurrently and deliver their work as a
branch instead of editing the user's working copy.

Opt in per resource with execution_mode=worktree (default stays in_place,
so existing resources are unchanged). Requires a git repo with at least one
commit; a non-git directory fails with an actionable message rather than
silently falling back to serial execution.

Design points that make the mode safe to use:

- The agent starts from what the user sees, not HEAD. Tracked edits are
  captured with `git stash create` (which writes a commit object without
  touching the index, working tree, or stash list) and replayed into the
  worktree; untracked non-ignored files are copied, bounded at 2000 files /
  200MB with the skipped count logged at warn level.
- That replayed state is committed as a baseline, so "did this task change
  anything?" has an exact answer and `git diff <baseline>..<branch>` is
  precisely the agent's work.
- Nothing is silently discarded: leftovers are committed to the branch
  before the worktree is removed, including when the task fails partway.
- A task that changes nothing leaves no branch behind.
- Sidecars stay out of the branch by running the existing
  CleanupRuntimeConfig + CleanupSidecars pass over the worktree before
  finalizing. The .git/info/exclude approach repocache uses is not usable
  here: a linked worktree resolves info/exclude to the user's own common git
  dir, so it would change what `git status` hides in their checkout.
- Worktree-mode env roots do not get the local_directory GC exemption, so
  they age out normally instead of accumulating one env root per task.
- `git worktree prune` on create self-heals registrations orphaned by a
  crashed daemon.

Adds --execution-mode to `project resource add/update`, and updates the
docs and the projects-and-resources built-in skill.

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): close data-loss and isolation gaps in worktree mode (MUL-5707)

Review follow-up on #6759. Six issues, all on failure or skew paths.

1. Preserve work when the final commit fails. Finalize logged a warning and
   then ran `git worktree remove --force`, destroying the agent's changes.
   commit.gpgSign with no key available to the daemon triggers this, and
   --no-verify does not disable signing. Finalize now returns an error and
   deliberately keeps the worktree — it stays registered, so `git worktree
   list` points at the work — and the daemon fails the task instead of
   reporting success with a branch that is missing the changes.

2. Fail closed on an unsupported execution_mode. Treating an unknown mode as
   in_place was wrong: execution_mode is how a user asks for ISOLATION, so
   the "safe" fallback would edit the working copy they asked us to stay out
   of. An unrecognised mode now fails the task naming the version skew.

3. Keep Codex conversations alive across turns. The per-issue session store
   was keyed on LocalWorkDir != "", so worktree tasks — which also never
   reuse an env root — got a fresh empty CODEX_HOME per turn and silently
   lost the thread. Covered by a two-turn regression test that fails without
   the fix.

4. Fail closed on an incomplete snapshot. A failed stash create/apply, a
   failed untracked copy, or hitting the copy bounds all degraded to "agent
   sees HEAD" with only a daemon log line. A half-replayed tree looks like a
   working checkout, so nothing downstream questions it while the agent reads
   different code than the user has. All of these now abort the prepare and
   tear the partial worktree back down.

5. Do not snapshot a directory mid-mutation. The same real path can be
   attached to another project as an in_place resource; snapshotting under a
   running in_place task captured a half-written tree plus that task's
   in-flight .agent_context/.multica sidecars, which then reached the branch
   and could feed the agent another issue's brief. Worktree prepare now takes
   the per-path mutex for the snapshot alone (released before the run, so
   tasks still overlap), and the untracked replay skips Multica's own
   sidecars.

6. Stop the test helper from masking regressions. gitRun skipped on any git
   failure, including post-finalize assertions, so a lost branch could report
   as a skip. Git availability is now checked once at repo setup; every later
   git failure is fatal.

Also rebased onto main, keeping both MulticaConfigRoot and LocalWorktree in
the Environment initialiser.

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): close sidecar-commit and wait-visibility gaps in worktree mode; gate worktree saves on daemon version (MUL-5707)

- runTask: register the sidecar-cleanup defer immediately after the Finalize
  defer. Previously two early returns (temp-dir setup, StartTask failure) sat
  between them, and on those paths Finalize auto-committed the sidecars
  Prepare had just written — delivering a branch whose only content is
  Multica's own runtime files (in_place: leaving them in the user's tree).
- runTask: give the worktree snapshot wait the same plumbing as the in-place
  acquire — waiting_local_directory status, resourceWaitTasks accounting, and
  a cancellation poller. The holder can be an hours-long in_place task; the
  wait was invisible and a user cancel went unnoticed until the 5-minute
  prepare timeout.
- isMulticaSidecarPath: match sidecar dirs as whole path segments at any
  depth, not only at the repo root. An in_place resource pointed at a
  SUBDIRECTORY leaves its sidecars below that subdirectory; replaying them
  committed another issue's brief into the delivered branch.
- handler: refuse saving execution_mode=worktree while the owning daemon's
  registered cli_version is below MinLocalWorktreeCLIVersion (422,
  daemon_version_unsupported). An old daemon json-skips the field and runs
  tasks IN PLACE — the daemon-side unknown-mode refusal cannot protect a
  daemon that predates the field, so the server has to. Residual (documented):
  a daemon downgraded after the save needs a claim-time gate.
- copyUntrackedFiles: read the -z listing raw instead of TrimSpace-ing it,
  which corrupted filenames beginning/ending with whitespace into paths that
  silently vanished from the replay.
- docs + skill: describe the save-time version gate; drop the stale claim
  that old runtimes fail the task themselves.

Co-authored-by: multica-agent <[email protected]>

* feat(views): UI switch for local_directory execution mode (MUL-5707)

Worktree mode shipped CLI/API-only, which for its target user — someone
running several tasks at once from Desktop — means it does not exist. This
adds the switch to the resources panel.

The choice is framed by what the user gets back, not by the mode
identifiers: "Edit this folder directly" (one task at a time, changes land
in your working copy) vs "Run in parallel, isolated" (tasks run at the same
time and each hands back an agent/… branch). `in_place` / `worktree` stay
visible as a secondary hint for anyone cross-referencing the CLI or docs.

Two entry points, matching where the decision is actually made:

- The add-a-folder flow now asks for the mode after the folder is picked and
  validated, instead of creating the resource and leaving the mode to be
  discovered later.
- Every local_directory row has a mode control. It is deliberately NOT gated
  on being the owning machine — switching the mode only rewrites a field, so
  it works from the web app too, unlike rename or the folder picker.

Both blocked states are handled where the user chooses, not after they
commit:

- Non-git folder: the desktop validator now reports whether the path is
  inside a git working tree (walking up for `.git`, so a subdirectory of a
  repo counts), and the option is disabled with that reason. Unknown — an
  older desktop build, or an existing row — stays permissive; the daemon
  re-checks authoritatively.
- Daemon too old: the version floor is mirrored from the server as
  MIN_LOCAL_WORKTREE_CLI_VERSION and checked against the runtime for the
  resource's OWN daemon, not the machine the browser happens to be on. A
  server rejection is rendered inline in the dialog rather than as a toast,
  so the dialog stays actionable.

A "Parallel" badge on the row is always visible (not hover-only): without it
there is no way to tell whether tasks on a folder edit it directly or hand
back a branch, which is the first thing anyone asks when a task queues.

Also adds `execution_mode` to LocalDirectoryResourceRef — the field existed
server-side but had no TypeScript counterpart — and brings the zh/ja/ko
project-resources docs in line with the English ones, which were the only
locale updated when the mode landed.

Co-authored-by: multica-agent <[email protected]>

* fix(daemon,handler): close worktree isolation and delivery gaps (MUL-5707)

Second review round. Five must-fix items, all on paths where the mode's
promises could still break.

1. A failed sidecar cleanup no longer ships Multica's own files as the
   user's branch. The cleanup errors were logged and Finalize then ran
   `git add -A`, so a permission/disk/lock failure could deliver a branch
   whose entire content was the runtime brief and issue context. Worktree
   mode now escalates a cleanup failure to an abort: nothing is committed,
   the worktree is kept for inspection, and the task fails with the
   preserved path. In place it stays a warning — no commit, no branch, and
   the leftovers are visible in the user's own tree.

2. The untracked replay no longer drops files silently. `Lstat` failures
   and symlinks were `continue`d without counting, so the caller read a
   partial snapshot as complete. Both now count as skipped, which the
   existing fail-closed check turns into a refused prepare. Non-content
   entries (sockets, FIFOs, devices) stay uncounted deliberately.

3. Prepare rolls back a worktree it created. Any later failure — context
   files, provider home, MCP config — returned without an Environment, so
   nothing downstream knew a worktree existed, leaving a registration and
   a branch no task ever ran in. A deferred Discard now runs unless Prepare
   reached its successful return.

4. The daemon-version gate moved to where it can actually hold. Save-time
   validation cannot see a machine downgraded afterwards, and an old daemon
   json-skips execution_mode entirely — it would run the task in place,
   editing the working copy the user asked to isolate. Claims are now
   checked against the runtime that is actually claiming, and a blocked
   task is cancelled with a reason rather than left to be redelivered to
   the same too-old runtime forever.

5. branch_name is delivered end to end. The daemon sent it, but
   TaskCompleteRequest had no field so it vanished at decode, and the
   failure path never sent it at all. It now has a column, travels on both
   terminal callbacks, and renders as a copyable row in Run details. The
   failure path matters most: worktree mode commits the agent's leftovers
   before tearing the worktree down, so a run that died partway still has
   work worth finding.

Docs (all four locales) and the projects-and-resources skill are updated
for the claim-time gate and the branch surface.

Co-authored-by: multica-agent <[email protected]>

* fix(handler,daemon): migration renumber, cancel reasons, branch on every terminal path (MUL-5707)

Third review round.

1. Rebased onto main and renumbered the migration to 285 — main had taken
   273 and is now at 284. sqlc regenerated on top of the rebased queries;
   the conflicts were confined to generated files.

2. The claim gate now persists why it cancelled. It called CancelTask,
   which writes only status='cancelled', and returned the reason as a 4xx
   that reaches the daemon's log and nothing else — batch claim drops the
   task from its response entirely. A run refused for a durable reason the
   user has to act on cannot present as a bare "cancelled", so this adds
   CancelAgentTaskWithReason and records error + failure_reason on the row.

3. branch_name now survives every terminal path, not just the two direct
   callbacks. Three were dropping it:
   - the context-exhausted complete→fail reroute did not copy it;
   - the daemon's complete→fail fallback (server rejected the complete,
     agent had succeeded) did not set it;
   - both cancellation paths discarded the result wholesale, even though
     the worktree is finalized — and its work committed — before the daemon
     learns of the cancel. The branch now rides the cancel-ack and is
     recorded by SetAgentTaskBranchName, which never overwrites a name a
     terminal callback already wrote.

4. Rollout/rollback compatibility. Isolation is enforced by two server-side
   gates, so a daemon downgraded at any point after the resource was saved
   is caught. The gap they cannot close is rolling the SERVER back past this
   change while sub-floor daemons are still connected: an old server has no
   gate and an old daemon ignores execution_mode. That constraint is now
   documented where an operator will meet it (the migration and the docs),
   and the version floor is pinned across server and frontend by test so the
   two gates cannot silently drift apart.

Co-authored-by: multica-agent <[email protected]>

* fix(server,views): route claim-gate cancels through TaskService, make cancel reasons visible, harden cancel-ack branch delivery (MUL-5707)

Closes the fourth-round review findings on worktree mode's cancellation
chain:

- Claim gate: cancel through a new TaskService.CancelTaskWithReason instead
  of a raw CancelAgentTaskWithReason query, so a gate-cancelled task gets the
  full cancellation side effects — audit capture, chat settle, agent status
  reconcile, task:cancelled broadcast, NotifyTaskFinished. Added REAL claim
  integration tests: singular claim (422 + row cancelled with reason), batch
  claim (task omitted + row cancelled with reason), and a current-runtime
  control.
- Cancel reasons are now user-visible: execution log rows, agent activity
  rows, and the transcript dialog label a SERVER-cancelled task (persisted
  error/failure_reason) like a failed one, with the actionable text on hover
  and a Reason row in Run details. User-initiated cancels stay a plain
  "Cancelled". New cancelReasonLabel helper + local_directory_error label;
  details_reason locale key in en/zh-Hans/ja/ko; widened
  AgentTask.failure_reason to the open string the wire actually carries.
- Cancel-ack is now a terminal delivery: the daemon retries it on transient
  errors (same schedule as complete/fail), the server fails LOUD (5xx) when
  persisting the branch/error fails instead of warn-and-200, and re-broadcasts
  task:cancelled after a successful write so clients that already refetched a
  branchless row pick up the delivery. Replays are idempotent (never
  overwrite).
- Cancel + Finalize-abort no longer swallows the preserved-worktree pointer:
  the finalize error is wrapped in worktreePreservedError (joined onto an
  existing run error, never displacing it), extracted on the cancel path, and
  delivered via the ack into agent_task_queue.error through the new
  SetAgentTaskErrorIfEmpty query — never overwriting a reason recorded by a
  fail callback or the claim gate. Other run errors on a cancelled run stay
  discarded as before: they are expected noise.
- Docs: the self-hosted rollback constraint callout now exists in zh/ja/ko,
  not just English.

Co-authored-by: multica-agent <[email protected]>

* fix(handler): guard cancel-ack writes to cancelled rows; requeue when the claim gate's cancel fails (MUL-5707)

Fifth-round review fixes:

- SetAgentTaskBranchName / SetAgentTaskErrorIfEmpty now carry a
  status='cancelled' CAS. The daemon acks on EVERY terminal status it
  observes, so a late or replayed ack from a stale run could previously
  backfill its branch or preserved-path error into a completed/failed row
  whose own callback — the authoritative channel — recorded nothing, without
  any event invalidating the change. A CAS-refused write is a deliberate
  no-op (still 200: nothing to retry), and the rebroadcast stays guarded by
  the same status check. Regression tests: a late ack against a completed row
  with empty fields and against a failed row leaves both untouched.
- When the worktree claim gate's CancelTaskWithReason fails, the claimed row
  no longer strands in dispatched (daemon refused, nothing to release it but
  stale reclaim). The gate requeues via RequeueTaskAfterClaimFailure — the
  next claim re-runs the gate and retries the cancel — and reports 5xx so the
  singular daemon reads a transient server problem, not a claim refusal; the
  batch path omits the task as before, now with the row requeued. Fault
  injection tests (row-scoped trigger raising on the cancel UPDATE while
  letting the requeue through) cover singular and batch.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Bohan-J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-13 14:52:01 +08:00