mirror of
https://gh-proxy.org/https://github.com/klinsmaya/multica.git
synced 2026-11-09 09:56:47 +08:00
admin-workspace-create2
16
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Younes Beriane
|
262dcb5fa7 |
MUL-7816 fix(taskfailure): classify a 403 usage limit as provider quota, not auth (#8967)
* fix(taskfailure): classify a 403 usage limit as provider quota, not auth Kimi Code reports an exhausted usage window as HTTP 403. The bare 403 auth rule ran before the quota rule, so the failure was filed as provider_auth_or_access. Match the usage-limit witness before the auth rule. The concurrent-request-limit case still comes first, and a plain 403 still classifies as auth. Fixes #8965 * fix(taskfailure): upgrade old-daemon 403 usage limits and beat the token rule Move the usage-limit check into a shared isUsageLimit403 helper that requires HTTP 403. Classify checks it right after the concurrent-request case, so an access-token-prefixed 403 usage limit no longer lands in context_overflow. NormalizeDaemonReason upgrades the same message from older daemons (context_overflow, provider_auth_or_access, unknown, agent_error) to provider_quota_limit. |
||
|
mrlonely
|
9d3613653e |
MUL-7164 fix(tasks): classify concurrent request rejections (#8200)
A provider rejection carrying the concurrent request limit witness is classified as provider_capacity_or_rate_limit ahead of the broad token/context and generic 403/auth rules, so members are no longer told to sign in again or reduce context for a transient concurrency rejection. NormalizeDaemonReason upgrades the same narrow witness from older daemons. Classification only: retryableReasons, retry delays and task creation are unchanged, and a regression pins provider_capacity_or_rate_limit as non-auto-retryable. The auto-retry proposal from #8180 is deliberately not included. Split from #8180 after review. |
||
|
mikyan
|
fe59262188 |
feat(agent): add Huawei Cloud CodeArts support (#6985)
* feat(agent): add Huawei Cloud CodeArts support * fix(migrations): resolve CodeArts prefix collision * fix(agent): address CodeArts review feedback * fix(agent): tighten CodeArts integration scope * fix(agent): resolve CodeArts follow-up review blockers * fix(agent): redact CodeArts launch arguments * docs: sync supported runtime counts * fix(agent): own CodeArts model probe process tree |
||
|
LinYushen
|
3b1018e9c9 |
Revert "MUL-6016: feat(task): retry exact Codex capacity failures (#6747)" (#7586)
This reverts commit
|
||
|
|
8fda4f22d5 |
MUL-6016: feat(task): retry exact Codex capacity failures (#6747)
* docs: record Codex capacity retry design * docs: add Codex capacity retry implementation plan * fix(taskfailure): recognize exact Codex capacity error * feat(task): retry exact Codex capacity failures * feat(server): configure Codex capacity retries * docs: expose Codex capacity retry setting * fix(task): align capacity retry comments --------- Co-authored-by: cuijs <[email protected]> Co-authored-by: Claude <[email protected]> |
||
|
|
acecb7dd60 |
fix(pi): keep turn error when process exits 1 [MUL-6585] (#7451)
* fix(pi): keep turn error when process exits 1 Pi (and pi-print-clean-exit) exits 1 after stopReason=error. The backend then overwrote lastTurnError with "pi exited with error: exit status 1", so Classify filed a LiteLLM Connection error / Request timed out as non-retryable process_failure. Preserve the provider message and classify those strings as provider_network so auto-retry can resume the session. * fix(taskfailure): scope Pi network error matching Co-authored-by: multica-agent <[email protected]> --------- Co-authored-by: buihongduc132 <[email protected]> Co-authored-by: Sol-Boy <[email protected]> Co-authored-by: multica-agent <[email protected]> |
||
|
Bohan Jiang
|
4d5a679b9b |
fix(openclaw): unblock the OpenClaw runtime on slow hosts (MUL-6321) (#7142)
Fixes #7112. Task preparation ran `openclaw config file` and `config get agents.list --json` with a hardcoded 5s deadline; on a 2020 Intel MacBook Pro those take 8.4-10.9s and 4.3s every time, so the OpenClaw runtime was unusable there with no way to raise the limit. - Deadline 5s -> 30s, overridable via MULTICA_OPENCLAW_CLI_TIMEOUT / backends.openclaw.cli_timeout, clamped to [1s, 60s] so the worst case (4 serial calls) stays inside the 5-minute preparation budget. - Config discovery cached per daemon profile (on disk, since preparation runs in a one-shot helper process), invalidated by binary identity, OpenClaw path env, active config stat, and a 60s TTL. Only complete successes are cached; any cache fault degrades to a live CLI run. - The timeout is classified structurally into a new platform-side, non-retryable reason (runtime_cli_timeout) instead of being text-matched into agent_error.provider_network, which showed "check your network" copy and auto-retried a deterministic local failure. Copy added for web/desktop/mobile; a NormalizeDaemonReason shim upgrades reports from older daemons. |
||
|
|
f51ee62b53 |
MUL-6164 fix(agent): diagnose an unrunnable agent CLI instead of leaking "exec format error" (#6963)
* fix(agent): diagnose an unrunnable agent CLI instead of leaking "exec format error" (MUL-6164) npm-distributed agent CLIs ship a placeholder text stub at their bin entry point and rely on a postinstall script to overwrite it with the platform-native binary. npm 12 blocks install scripts unless allowlisted, pnpm 10 needs `pnpm approve-builds`, and --ignore-scripts / --omit=optional do the same — all of them leave the stub in place with its executable bit set. exec.LookPath accepts it, execve rejects it, so the CLI reads as installed everywhere the daemon looks and only fails at launch with a bare "start claude: fork/exec <path>: exec format error". Add ExplainExecError: on ENOEXEC it names the path, the owning npm package, and the exact repair command read out of that package's own scripts.postinstall. The executable path is recovered from the error itself, so this is one provider-agnostic call at each boundary — the shared version probe and the daemon's single backend.Execute site — rather than a wrap in every backend. claude and opencode ship this layout today; a CLI added later is covered without a code change. Also classify "exec format error" as runtime_missing_executable: a stub that cannot exec means this host has no usable CLI, and that reason already tells the daemon a fresh-session retry cannot help. Co-authored-by: multica-agent <[email protected]> * fix(agent): keep the ENOEXEC diagnosis to one copy (MUL-6164 review) The openclaw preflight explained the error, then returned it through openclawBackend.Execute into the daemon's launch boundary, which explained it again — %w keeps ENOEXEC reachable, so the second pass fired and the user got the same paragraph twice. Drop the openclaw wrap: its error already travels through the daemon boundary, so the extra site bought nothing. Make ExplainExecError idempotent as well, so the next nested boundary cannot reintroduce the same regression silently. Also quote the package root properly in the repair command — it is meant to be pasted into a shell, and a path holding a quote or a space has to survive that. The chat copy for runtime_missing_executable said the CLI "couldn't be found / check it's on PATH", which is the opposite of this failure: the file is right there on PATH and cannot run. Reworded to cover both across all four locales; the compact labels ("Runner CLI not installed") are left for the follow-up that reworks the reason's presentation. Co-authored-by: multica-agent <[email protected]> --------- Co-authored-by: Bohan-J <[email protected]> Co-authored-by: multica-agent <[email protected]> |
||
|
|
85045228a1 |
MUL-6102: fix(daemon): surface which HERMES_HOME a Hermes task actually read (#6892)
* fix(daemon): surface which HERMES_HOME a Hermes task actually read A Hermes task runs against a per-task HERMES_HOME overlay, seeded from a source home the daemon resolves from its own process environment. When that resolution lands on a home holding no config.yaml, the overlay is built anyway from nothing: the derived config keeps only skills.external_dirs, so the user's provider and credentials are gone and Hermes refuses every session/new with "No LLM provider configured". Nothing on that path said so. The overlay was seeded silently, no line logged the resolved source home, and the failure surfaced Hermes' own remedy -- run `hermes model` -- which edits whichever home the user's shell points at, not the one the task read. GH #6872 is what that costs: eight documented workarounds, none of which could have worked, and a root cause filed against an unrelated resume bug. Three additions, no behaviour change: - Warn when the source home is missing, or exists without a config.yaml. Kept a warning rather than a hard failure because config-less is a legitimate setup: an image that ships Hermes and injects OPENAI_API_KEY through the environment resolves a provider fine. - Log the resolved source home once per task, at Info. - Append the overlay and source homes to a "no LLM provider configured" failure, so the task itself names the directory that was read. The annotation runs after every classifier has read the error text. It embeds two user-chosen paths and Classify matches substrings anywhere, so annotating earlier would let a directory name pick the failure_reason -- a home under /srv/token-cache/limit/ satisfies rule 1 (context overflow), which is evaluated before rule 2 (missing config) and feeds retry eligibility and resume blacklisting. The phrase rule 2 already matched is now a shared const so the new predicate and the classifier cannot drift apart. Co-authored-by: multica-agent <[email protected]> * fix(daemon): keep the Hermes hint out of resume-guard input Review caught that the annotation did not stop at the daemon. It travels to the server as the task's error, is persisted in agent_task_queue.error, and is re-scanned there for the life of the row by service.ResumeUnsafeFailure and by the ILIKE/regex guards in GetLastTaskSession / GetLastChatTaskSession. Both interpolated paths are user-controlled -- HERMES_HOME comes from the agent's custom_env, the overlay root from MULTICA_WORKSPACES_ROOT -- so a legal directory name got a vote on session recovery. Measured, with a source home under /srv/400-invalid_request_error/: raw error ResumeUnsafeFailure = false annotated with the paths ResumeUnsafeFailure = true annotated with this hint ResumeUnsafeFailure = false That true is a healthy chat session pointer being cleared on a failure that has nothing to do with it, so "no resume semantics change" was not true as written. The hint is now fixed prose with nothing interpolated, and names the daemon log line that does carry the paths. This removes the vector rather than containing it: no wire or schema change, and it is correct against old servers too, so there is no mixed-version window where an un-upgraded server misreads a new daemon's text. Tests cover the boundary the previous ones missed: classification AND service.ResumeUnsafeFailure agree with the unannotated text across five real failure shapes, plus a check against the SQL guard phrases, which no unit test can execute. Also softens the missing-config warning, which asserted the task runs without credentials when the environment may well supply them -- the config-less-but-working setup this PR deliberately keeps supported. Co-authored-by: multica-agent <[email protected]> --------- Co-authored-by: Bohan-J <[email protected]> Co-authored-by: multica-agent <[email protected]> |
||
|
Jamison-J
|
d582dca062 |
MUL-6025: fix(daemon): resolve Windows installer junctions (#6802)
Windows standalone Codex exposes its installer bin directory as an IO_REPARSE_TAG_MOUNT_POINT junction, which filepath.EvalSymlinks cannot traverse, so the daemon launched the shim path and Codex could not find its sandbox helpers. Discovery now keeps the stable entry point and the final target is resolved per launch via GetFinalPathNameByHandleW, so installer retargets take effect while old releases remain installed. Resolution failures fail closed on both the initial and rediscovery paths. Refs #6762 |
||
|
|
37e2b07094 |
MUL-5831 fix(agent): fail opencode runs that end on an empty step (#6545)
* fix(agent): fail opencode runs that end on an empty step (MUL-5831) `opencodeBackend.processEvents` required a positive terminal signal but only failed closed on two shapes: EOF while a step was still open, and a step that finished with reason "tool-calls" whose continuation never started. A final step that opens, emits only a reasoning part, and closes with reason "unknown" and an all-zero token block matches neither, so `opencode run` exited 0 and the daemon reported `completed` with an empty `agent_error` — a run with no deliverable that leaves the issue stalled until something re-wakes the agent. Zero input tokens means the provider round-trip never happened; that is a dead stream wearing a clean finish. Track whether each step produced anything at all — text, a tool call, or any non-zero token counter — and fail closed when the run ends on a step that produced none of them. The criterion is deliberately "this step produced nothing", not "the run produced no text", so a task whose only deliverable is a tool side effect stays green. The finish reason is not consulted: a missing or unrecognised reason must stay terminal for protocol compatibility, and voidness is orthogonal to it. The run now enters the existing failure path with a diagnosable error, so the task's retry rules apply instead of a silent false-green completion. Refs #6522 — this fixes the first of its three symptoms; see the PR body. Co-authored-by: multica-agent <[email protected]> * fix(agent): count every opencode usage field as round-trip evidence (MUL-5831) The void-step guard claimed "any non-zero token counter keeps the step productive", but only checked input, output and cache. OpenCode's protocol keeps reasoning and the aggregate total in fields of their own and reports cost as a sibling of the token block, and none of the three were parsed at all — so a real step that landed with reasoning or cost positive while input and output happened to be zero would be called void and fail a healthy run. Parse reasoning, total and cost, and move the check into stepReportedUsage so the criterion is stated once. Reasoning and total are read as evidence only: total is derived, and TokenUsage has no reasoning bucket, so folding either into the usage accumulator would change billing figures rather than fix a bug. Also renames the guard's error text to "no reported usage", which is what it now means. Co-authored-by: multica-agent <[email protected]> * fix(taskfailure): make opencode stream-ended failures retryable (MUL-5831) Turning a false-green completion red only helps if the failure then retries. It did not: none of the three errors the OpenCode terminal-signal guard raises was on the retry allowlist. The two "terminal signal" variants matched rule 13's "signal" substring by accident and landed in agent_error.process_failure; the new empty-step one fell through to agent_error.unknown. Either way the task died on the first attempt and max_attempts never applied. All three mean one thing — the provider stream died and `opencode run` still exited 0 — which is agent_error.provider_network by definition. Route them there on the shared "opencode stream ended" prefix. That bucket is retryable and resume-safe, so the retry child inherits the session and continues the truncated conversation instead of redoing the work already paid for. Covered end to end: the classifier cases, and a service-level test that walks each guard error through Classify into retryEligible — retrying with an attempt left, and still terminating at the ceiling so a deterministically broken provider cannot loop. Co-authored-by: multica-agent <[email protected]> * fix(taskfailure): upgrade legacy opencode stream-ended reasons (MUL-5831) Rule 7 only decides where these failures land when THIS server classifies them, and FailTask classifies only when the daemon sent no reason at all. An installed daemon predating that entry reports a non-empty agent_error.process_failure instead — its rule 13 matching the word "signal" in "terminal signal" — so the empty-reason branch skips it and the run stays off the retry allowlist. The fix reached only hosts that happened to update, which is the installed-daemon cadence problem NormalizeDaemonReason exists for. Upgrade process_failure / unknown / coarse agent_error to provider_network when the error opens with the guard's own prefix. Wider than the context-overflow rule above it on purpose: that one leaves refined reasons alone because a phrase somewhere inside an error blob says less than the bucket an earlier rule picked, whereas this witness is the guard's message from its first character, so the old bucket cannot be describing a better-identified cause. The retry test becomes a matrix over both wire shapes — current daemon (server classifies) and each legacy reason — since the previous version called Classify directly and never crossed the normalize boundary where the gap was. A negative case pins the prefix: an unrelated crash that merely mentions the phrase keeps the daemon's label. Co-authored-by: multica-agent <[email protected]> --------- Co-authored-by: Bohan-J <[email protected]> Co-authored-by: multica-agent <[email protected]> |
||
|
|
79c5832e1d |
MUL-5708 fix(taskfailure): classify response-side context-window overflow (#6366)
* fix(taskfailure): classify response-side context-window overflow (MUL-5708)
Claude Code 2.1.x reports an exhausted context window on the response,
not as a 400 on the request: the turn ends with stop_reason
"model_context_window_exceeded" and the CLI prints
API Error: The model has reached its context window limit.
That string carries none of the phrases rule 1 matched and no "token",
so it classified as agent_error.unknown. Unknown is absent from the
resume blacklists (resumeUnsafeFailureReason, GetLastTaskSession,
GetLastChatTaskSession), so the over-full session stayed pinned as the
resume pointer for the (agent, issue) pair and every later comment on
the issue resumed the same transcript and overflowed again.
Match the CLI copy and the raw stop reason so the failure lands in
agent_error.context_overflow, which those blacklists already exclude —
the next comment then starts from a fresh session instead of replaying
the overflow.
Co-authored-by: multica-agent <[email protected]>
* fix(taskfailure): upgrade an old daemon's catchall on context overflow (MUL-5708)
Installed daemons update on their own cadence, and FailTask only
re-classifies when the caller supplied no reason. A daemon whose rule 1
predates the response-side wordings reports agent_error.unknown, which
is on no resume blacklist — so until every host updates, the over-full
session stays pinned as the (agent, issue) resume pointer and every
later comment replays the same overflow. One un-upgraded host means a
permanently stuck issue, not just a missing label.
Recognise the two witnesses server-side in NormalizeDaemonReason, next
to the MUL-5370 skill-bundle rule it mirrors, so the retirement lands
the moment the server deploys. The accepted legacy set is narrower than
that rule's: only the catchall and the pre-MUL-1949 coarse agent_error.
A refined reason means the old daemon matched an earlier rule on the
same text, which says more about what ended the run than a witness
appearing somewhere in the blob does.
The witnesses move into one shared var so Classify and the normalizer
cannot drift apart.
Co-authored-by: multica-agent <[email protected]>
---------
Co-authored-by: Bohan-J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
|
||
|
|
c271f80999 |
MUL-5370 fix: label stalled skill-bundle downloads, align failure-reason copy with the backend taxonomy (#6001)
* fix(daemon): label stalled skill-bundle downloads and make them retryable A skill bundle that could not be downloaded during task preparation surfaced as the bare string "resolve skill bundles: context deadline exceeded". taskfailure.Classify has no rule for a Go context deadline, so it landed in agent_error.unknown — a bucket that is NOT on the server's retry allowlist. A transient stall therefore became a terminal chat failure carrying a label nobody could act on, and the failure was invisible on the Usage page's Errors breakdown. (MUL-5370) - Add the platform-side reason skill_bundle_unavailable and put it on retryableReasons. Retrying is cheap and safe: the agent process never started, and bundles that did arrive are already cached on disk, so successive attempts converge. - Carry a sentinel error from the resolve loop so the reason is derived structurally rather than by matching the wrapped transport error's text, and name the skill, its declared size and the elapsed wait in the wrap — enough to tell "this bundle is too big for the link" from "the link is dead" without reading daemon logs. - Normalise the wire shape an OLD daemon produces (a non-empty catchall plus the previous "resolve skill bundles:" wrapper) on the server side. Installed daemons upgrade on their own cadence, and FailTask only classifies when the caller supplied nothing, so without this the fix would reach only hosts that happened to update — while the un-upgraded hosts most likely to be hitting the bug kept failing terminally. - Teach Classify about "deadline exceeded" and net/http's "Client.Timeout exceeded while awaiting" so any other Go-side deadline that reaches it as text stops falling into the unknown bucket too. - Backfill historical rows in both agent_task_queue and chat_message. Scoped to agent_error.unknown alone — the old wrapper string postdates the in-flight classifier by three weeks, so no row carrying it can hold the legacy coarse value — which keeps the down migration an exact inverse. Co-authored-by: multica-agent <[email protected]> * fix(chat): give chat its own failure copy for the refined reasons #5991 rebuilt the operator-facing failure labels around an open wire string with a raw-value fallback, but the chat bubble kept its own exact-key lookup against the six coarse values from migration 055. So all 14 agent_error.* values still missed and rendered the generic "Something went wrong and the agent couldn't finish replying" — the classification the backend had already computed was discarded at the last step, and that is the message the MUL-5370 reporter saw. - Add resolveFailureReasonKey in packages/core: exact match, else degrade an `agent_error.*` value to its family, else undefined. A reason newer than the shipped client now lands on the family line instead of the fallback. - Rekey the chat copy map by wire value and route it through the helper. Chat deliberately degrades to friendly copy rather than adopting the operator surfaces' raw-value fallback: it is read by the person who just sent a message, and the raw error is one click away under the collapsible. - Add refined chat copy (en / zh-Hans / ja / ko) only where it can say something the family line can't — a different next step: network, auth, quota, rate limit, context overflow, missing/outdated CLI, skill download. - Give skill_bundle_unavailable a label on the web and mobile surfaces and a class on the Usage page's Errors breakdown (runtime — the operator response is "check the daemon's link to Multica", the provider is not involved). - Mobile's two label maps were still coarse-only for the same reason; rekey them by wire value and fill in the refined taxonomy. Co-authored-by: multica-agent <[email protected]> --------- Co-authored-by: Bohan-J <[email protected]> Co-authored-by: multica-agent <[email protected]> |
||
|
|
6dba74c3c1 |
fix(task): auto-retry transient "Connection closed mid-response" like chat (MUL-4910) (#5565)
* fix(task): auto-retry transient provider stream cut like chat (MUL-4910) Claude Code's "API Error: Connection closed mid-response" is a transient network cut. In unattended issue runs it fell through to agent_error.unknown / process_failure — neither in retryableReasons — so the task terminated. Interactive chat only appeared resilient because the CLI's own in-process retry usually recovers first; there is no chat-specific retry in Multica. Both paths share the same finalizeStreamResult -> Classify -> retryableReasons pipeline. - classify: route "connection closed" / "mid-response" to agent_error.provider_network, before the exit-status rule so the "exited with error: exit status N" variant also lands here. - retry: add provider_network to retryableReasons. It is resume-safe, so the retry child inherits the session and continues the truncated conversation instead of restarting. Tests cover the new classification (incl. exit-status variant) and the retry/resume flags. Note: mirror the substrings into the MUL-1949 offline backfill SQL to keep in-flight and historical taxonomies aligned. Co-authored-by: multica-agent <[email protected]> * feat(task): defer provider_network's final retry ~5s — three-tier (MUL-4910) Follow-up to the immediate-retry fix: make the connection-closed retry a three-tier schedule — first run + immediate retry + one retry deferred ~5s — so a blip that survives the immediate retry gets a short cooldown before the final attempt instead of firing back-to-back. Reuses the existing deferred/fire_at primitive (the comment-routing escalation mechanism) rather than adding new infrastructure: CreateRetryTask gains an optional fire_at; when set, the child is inserted 'deferred' and the existing PromoteDueDeferredTasksForRuntime sweeper — already run promote-first on every claim poll — flips it to 'queued' at fire_at. No migration, no claim-query change, no daemon change. - retryAttemptCeiling: raise provider_network's ceiling to 3 (other reasons keep max_attempts=2); applied in both retryEligible and MaybeRetryFailedTask's budget pre-check so the primary and sweeper paths agree. - retryDelayForAttempt: only provider_network's final attempt is deferred (5s); every other retry — including provider_network's first — stays immediate. - FailTask + MaybeRetryFailedTask pass fire_at and skip the queued broadcast/notify for a deferred child (promotion emits them at fire time). Timing: the deferred child fires on the first claim poll at/after fire_at, so >= 5s; on an otherwise-idle runtime it can stretch to the poll interval — the same behaviour deferred escalations already have. Tests: pure schedule/eligibility coverage (TestProviderNetworkRetrySchedule) plus a DB test asserting fire_at controls deferred-vs-queued, attempt=3, and resume-safety. Co-authored-by: multica-agent <[email protected]> * fix(task): persist reason-aware retry budget; respect max_attempts=1 disable (MUL-4910) Addresses the pre-merge review must-fix: retryAttemptCeiling raised provider_network to 3 unconditionally, which (1) overrode the max_attempts=1 "auto-retry disabled" contract and (2) persisted a self-contradictory child (attempt=3, max_attempts=2) that leaks to the task API, so a naive attempt < max_attempts consumer would misjudge the budget. - retryAttemptCeiling now returns taskMaxAttempts unchanged when it is <= 1 (disabled stays disabled) and only ever WIDENS otherwise (max(col, 3) for provider_network) — a higher configured budget is kept. - CreateRetryTask takes an optional max_attempts; FailTask and MaybeRetryFailedTask write the reason-aware effective ceiling into the child so the whole retry chain self-describes (attempt=3, max_attempts=3). NULL inherits the parent column, so non-provider_network reasons are unchanged. Tests: - pure: ceiling widens to 3, keeps a higher budget, and never revives a disabled (max_attempts=1) task; eligibility rejects the disabled case. - DB (end-to-end FailTask): default budget → deferred final child at attempt=3/max_attempts=3; first failure → immediate child; max_attempts=1 → no child. Plus CreateRetryTask persists the passed budget / inherits on NULL. Co-authored-by: multica-agent <[email protected]> --------- Co-authored-by: Bohan-J <[email protected]> Co-authored-by: multica-agent <[email protected]> |
||
|
Rusty Raven
|
7356b56a10 |
fix(taskfailure): anchor HTTP status-code matches to digit boundaries (#5275)
Anchor the 401/402/403/429/529 HTTP status-code matches to digit boundaries so embedded numbers (e.g. "402913 tokens", "15290ms") no longer misclassify process/unknown failures as provider errors and skew failure observability. Mirrors the existing 5xx anchoring (providerHTTP5xxRe). Fixes #5271 MUL-4422 |
||
|
|
10afd1af1b |
feat(server): introduce pkg/taskfailure classifier and switch in-flight failure_reason writes (MUL-2946) (#3693)
Lift MUL-1949's offline backfill failure_reason taxonomy into a shared in-flight classifier so the agent_task_queue.failure_reason column is written with refined values (provider_auth_or_access, context_overflow, provider_capacity_or_rate_limit, …) at write time rather than waiting on SQL backfill to re-classify after the fact. PR1 of the Grafana board plan in MUL-2328 — the upcoming PR2 reuses pkg/taskfailure.AllReasons() to pre-warm the Prometheus failure_reason label set. * server/pkg/taskfailure: new package with the canonical 21 Reason constants (7 platform-side + 14 agent_error.* sub-reasons), AllReasons() returning a defensive copy, IsAgentError() prefix check, and Classify(rawError) Reason mirroring the SQL CASE rules from MUL-1949 (db-boy's analysis). 100% statement coverage. * server/internal/daemon/daemon.go: route the 'agent_error' coarse fallback paths (StartTask error, runTask early-return error, CompleteTask permanent rejection, reportTaskResult default branch) and the executeAndDrain default error case (chained after classifyPoisonedError) through taskfailure.Classify so blocked / timeout / unknown-status results all carry a refined reason on the wire. * server/internal/service/task.go: FailTask classifies errMsg when the daemon-supplied failureReason is empty, eliminating the legacy COALESCE(.., 'agent_error') landing. * server/internal/daemon/poisoned.go: alias FailureReasonIterationLimit and FailureReasonAPIInvalidRequest to the canonical taskfailure constants. agent_fallback_message and codex_semantic_inactivity are pre-existing operational reasons not in the canonical 21 — kept as literals for now and revisited in a follow-up PR. Backfill SQL from MUL-1949 stays as the authoritative offline source of truth; this PR keeps the in-flight classifier in lock-step with the SQL CASE expression so historical and future rows share the same taxonomy. No behavior change for the platform-side reasons (queued_expired, runtime_offline, runtime_recovery, timeout, etc.) which already align with the canonical set. Co-authored-by: Eve <[email protected]> Co-authored-by: multica-agent <[email protected]> |