mirror of
https://gh-proxy.org/https://github.com/klinsmaya/multica.git
synced 2026-11-10 10:26:46 +08:00
On a self-hosted tenant a user sent a WeCom message at 18:31:17.7 while the
previous agent turn was finishing. The engine enqueued a task for it at
18:31:20.7 and cancelled the task 27ms later:
ERR chat claim: task-owned direct task has no user input; cancelling
task_id=bdc0947e chat_session_id=2fccd3c9 chat_input_task_id=bdc0947e
The database afterwards:
chat_message "挺好的 正常了" (18:31:17.726997) -> task_id = NULL
every other user message in that session -> task_id set
select count(*) from chat_message
where task_id='bdc0947e' and role='user' -> 0
The user's message was accepted, a task appeared and instantly vanished, and
no answer ever came. On WeCom the typing bubble also spun forever, because the
indicator only unsubscribed on task:failed, not task:cancelled.
Mechanism
Channel messages (Slack / Lark / WeCom) are written durably and unowned on the
ACK path; the 3s silence debouncer later fires one flush per session, and
EnqueueChatTask creates the task, stamps chat_input_task_id = id, and seals the
waiting messages into it with LinkUnownedChannelChatMessagesToTask. That seal
skipped any unowned user row that had a non-user row after it. The rule was
carried over from the pre-ownership trailing-message selector, and it does not
survive the debounce window: a predecessor completing inside that window writes
its reply AFTER the waiting message, so the reply to an already-sealed batch
became the boundary for a message it never saw.
18:31:04.6 user "测试下" unowned
18:31:07.6 task 6a4ae7f8 created, seals it
18:31:17.7 user "挺好的 正常了" unowned, 3s window arms
18:31:20.2 6a4ae7f8 completes, writes its reply <- newer than the message
18:31:20.7 window expires, task bdc0947e created, seal matches ZERO rows
18:31:20.7 claim finds no input, cancels bdc0947e
Every timestamp above is from the tenant's own tables. The claim guard at
internal/handler/daemon.go:2347 is right to fail closed on an empty input
batch; the batch should never have been empty. Its comment blamed a send path
that does not commit message and task together — SendDirectChatMessage does,
and the channel path does too. Neither was the problem. The seal predicate was.
Fix
The boundary is "already answered", not "something newer exists". A reply can
only be a reply TO a message if the turn that wrote it started AFTER the
message arrived. A turn already running when the message landed sealed its own
input before the message existed, so its reply — however late it lands —
answers an earlier batch and must not move the boundary. The seal now qualifies
each candidate boundary row by its turn's creation time, resolved through
COALESCE(chat_input_task_id, id) so an auto-retry clone is judged by the root
batch it inherited rather than by its own fresh id.
The check is kept, not dropped, and deliberately stays conservative: a non-user
row whose task is NULL or gone still counts as a boundary, so pre-ownership
transcripts are never swept into a new batch, and a message already stranded by
this bug is not resurrected into a turn minutes later — a following turn that
ran past it holds the boundary.
Tests
internal/handler/chat_channel_debounce_boundary_test.go drives the whole
failure: message in, claim, run, second message while it runs, predecessor
completes, window expires, claim. Before the fix:
--- FAIL: TestChannelChat_ReplyLandingInsideDebounceWindowStillAnswersTheNextMessage
chat_channel_debounce_boundary_test.go:134: claim = 500 {"error":"chat task has no user input"}
(task <nil>); task 2a67492b-cd61-45e6-afd6-156079971152 owns 0 user messages and is now "cancelled". The user's message was accepted and never answered.
internal/service/task_channel_seal_race_test.go pins the same boundary at the
seal, twice — the sequential ordering the tenant hit, and the tighter window
where the reply commits on another connection after the enqueue transaction
has begun (same raceInjectTxStarter the media-deferral race test uses), so
the seal statement's READ COMMITTED snapshot is the first to see it. Before
the fix:
--- FAIL: TestEnqueueChatTaskSealsMessageStrandedByPredecessorReply
task_channel_seal_race_test.go:78: waiting user message task_id = "", want the new turn "34b62bad-955c-46f0-99e9-2194999d2c10" — a predecessor reply that landed after the message must not seal it out
--- FAIL: TestEnqueueChatTaskSealsMessageWhenReplyCommitsDuringEnqueue
task_channel_seal_race_test.go:142: waiting user message task_id = "", want the new turn "989a83e4-e945-4b36-a5b7-69720449e08c"
The third, TestEnqueueChatTaskLeavesAlreadyPassedMessagesUnowned, passes both
before and after by design: it is the guard against over-correcting, and only
a fix that swept too much could break it.
The two transactions cannot interleave more finely than this: CompleteTask
takes chat_session FOR UPDATE and the task INSERT takes the FK's FOR KEY SHARE
on the same row. "Predecessor commits, then the flush runs" is the reachable
interleaving, and it is the one the tenant hit.
Claude-Session: https://claude.ai/code/session_01MKFf7wYM9Q2iaPtoFhG8MV
Co-authored-by: Claude Opus 5 <[email protected]>