Commit Graph
10 Commits
Author SHA1 Message Date
Bohan Jiang c923ace620 fix(agent): let omp (Oh-My-Pi) agents set a thinking level (MUL-7412) (#8464)
Agents on the built-in omp (Oh-My-Pi) runtime could not set a reasoning effort at
all. omp dispatches to the pi backend but was missing from providerThinkingEnums,
so the server rejected every value, and parseOmpModels discarded omp's per-model
effort catalog, leaving the picker hidden and the daemon's guard nothing to
validate against. `--thinking` is blocked in custom_args by design, so the only
workaround was folding the level into the model string (`devin/swe-2:max`).

What a catalog entry means is taken from can1357/oh-my-pi v18.2.0 rather than
inferred from pi, because two of pi's rules are inverted for omp:

- `models --json` emits `thinking` as an effort array or `null`, never an object.
- omp's Effort vocabulary excludes `off`, yet resolveThinkingLevelForModel returns
  `off` before clamping runs, so it is honoured for every reasoning model.
- `reasoning: true` with `thinking: null` means "reasons, no controllable dial";
  inferring levels there would offer efforts omp then clamps away.

So a reasoning model advertises `off` plus exactly the efforts the catalog listed,
a non-reasoning model gets no picker, and nothing is inferred. `auto` is
deliberately not exposed: omp keeps it as a session-level sentinel that is never
an Effort or ThinkingLevel.

An effort with no pinned model is refused rather than stored, because omp resolves
its own default role model at task time and clamps the level to what that model
supports — so a stored level would save cleanly, read back as set, and then run at
a different level. The refusal holds in all three layers: the inspector hides the
picker, CreateAgent/UpdateAgent return 400 (Update against the state the request
lands on, so clearing the model while an effort is carried over is caught too),
and ValidateThinkingLevelWith rejects before any catalog load, so a discovery
error cannot reach the daemon's fail-open path.

codex shares that execution constraint but stays out of the API-level refusal via
a separate narrower predicate: agents already hold an effort with an empty model,
and 400ing that pair would block unrelated edits to them. omp has no such history.

Closes #8458
2026-09-16 12:14:09 +08:00
a075e58b80 MUL-6961 feat(agent): discover Claude Code's model catalog from the CLI (#7917)
* feat(agent): discover Claude Code's model catalog from the CLI

Claude was the last first-party runtime answering "what models do you
have?" from a hand-maintained list. A static catalog cannot know what the
CLI in front of it can run, so every new Anthropic model opened a window
where Multica offered a model that 400s:

    API Error: 400 Claude Code 2.1.246 does not support this model;
    version 2.1.251 or newer is required.

Claude Code does have a discovery hook, just not on the command line: the
stream-json control protocol answers a `list_models` control request with
the rows its own /model picker renders. Ask the installed binary and the
window closes at the source — a model it cannot run is either absent or
flagged disabled, computed against the real CLI version and the logged-in
account.

Verified against 2.1.223/228/246/258. The three older builds return Fable
5.1 as a disabled row carrying "Update to 2.1.255+ to use Fable 5.1";
2.1.258 returns it as a normal pick. Those rows are the reason no
per-model minimum-version table is needed here: upstream already computes
one, including the remedy copy, which we forward verbatim and grey out.

No version gate guards the request. An unsupported subtype is answered,
not ignored — every build tested replies "Unsupported control request
subtype" in ~2s and exits 0 — so an old CLI costs one cheap round trip and
falls back. Gating would mean hand-maintaining exactly the kind of number
this change removes.

Notes:

- The static catalog stays, now as the fallback and as the server-side
  accept list. It is flagged Catalog.Fallback so a discovery failure is
  not pinned in the server's day-long cache (MUL-5549).
- Discovery replaces rather than merges, as with every other dynamic
  provider. The picker narrows to the CLI's own list; agents already
  pinned to an older model keep working, and manual entry still reaches
  them.
- Effort levels now come from each row instead of scraping `claude
  --help` and narrowing through the hand-kept claudeModelEffortAllow.
- Catalog ids keep the context-window tag, because `claude-opus-5[1m]` is
  what the CLI would really run for that row. That exposed a latent bug
  in two capability lookups, which normalized the query side but not the
  catalog side; both now normalize both, or a model would fail to find
  its own effort picker.

MUL-6961

Co-authored-by: multica-agent <[email protected]>

* fix(agent): keep unrunnable Claude models out of the selectable catalog

Review found the first cut carried "this CLI cannot run it" as a flag on
the model itself. That was wrong in a way a flag cannot fix.

Only the create-agent dropdown learned the flag. The inspector picker and
the agent builder read the same list and kept offering the row, so
editing an existing agent still reached the 400 this change exists to
prevent. And the part no amount of teaching fixes: an already installed
desktop client has never heard of the field, so it renders the row as an
ordinary model and persists a model id the CLI rejects — exactly what the
repo's mixed-version API rule forbids.

Unavailable models now travel in their own list, end to end: daemon
report, store, catalog cache, and the client result. `models` contains
only selectable entries. The inspector and the builder needed no changes
to become correct — they read `models`, and the rows are no longer in it
— and an old client is correct for the same reason. Both pickers render
the advisory rows through one shared component that takes no click
handler, so nothing can turn them back into controls.

Two more review findings:

- The default row was dropped when no sibling row named the same model,
  which an org-restricted install can produce. That is not one missing
  picker entry: ValidateThinkingLevelWith resolves an empty model — the
  common "follow the CLI default" configuration — through the entry
  flagged Default, and fails closed without one, silently discarding the
  user's effort on every such task. It is now materialised, keeping the
  row's own capabilities. The test that claimed to cover this asserted
  the opposite of its own comment; rewritten, plus a case proving an
  unrunnable default is never materialised.

- "One cheap round trip" for a CLI without list_models was one round trip
  PER TASK: every Claude task carrying a thinking_level loads the catalog
  before it starts, and cachedDiscovery refuses to memoise a fallback.
  The structured unsupported-subtype answer is now remembered per
  (command, CLI version) — a property of the binary, not a bad moment —
  so an old CLI is probed once. Transient failures stay uncached and
  retryable, and the version in the key makes an upgrade take effect at
  once rather than after a TTL. Detecting it costs ~0.01s against the
  ~1.7s it saves, and the fallback path already paid for it.

The discovery timeout keeps its 20s ceiling, now with the measurements
that justify it: 1.57-2.19s over five runs against 2.1.258, 1.64-1.79s
against 2.1.246. The ceiling is for a CLI that never answers, not a slow
one.

Verified against real binaries: 2.1.246 offers four selectable models and
one unavailable row carrying upstream's own upgrade hint, 2.1.258 offers
Fable 5.1 normally, a missing binary falls back to the flagged static
catalog.

MUL-6961

Co-authored-by: multica-agent <[email protected]>

* docs(agent): record the downgrade case as an accepted boundary

Dynamic discovery governs selection, and only selection. An agent already
configured with a model whose CLI is later downgraded still starts and
still gets the upstream 400 — discovery cannot reach a value already
persisted.

That is deliberate, not an oversight, so it is written down next to the
code that invites the wrong fix. Clearing the configured model turns a
legible error into a silent config change the user did not make;
substituting a runnable one is worse, because it succeeds using a model
nobody chose. The downgrade is rare and the upstream error is accurate,
so failing loudly at task start is a fine way to report it.

MUL-6961

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-09-02 15:48:38 +08:00
740ebdb8ef MUL-5991 feat(agent): enable ACP thinking_level for the hermes provider (jcode) (#6814)
* feat(agent): enable ACP thinking_level for the hermes provider (jcode)

The reporter confirmed from jcode source (v0.71.1 and v0.73.0) that its
advertised effort is genuinely applied, not just echoed: session/set_config_option
waits for an `effort_changed` ack, and the provider request carries
`reasoning.effort` upstream. That was the one open question blocking this
half of GitHub #6720.

The wrinkle is that `hermes` is one provider type over two unrelated
binaries with opposite answers here:

  - jcode advertises option id `reasoning_effort` under category
    `thought_level` and applies it. The shared parser already matches on
    category, so the id being unrecognised costs nothing and the id we
    send back is jcode's own.
  - Hermes Agent has no effort surface on ACP at all. Re-verified against
    v0.20.0 today by driving `hermes acp` directly: session/new still
    returns only `_meta`, `models`, `modes`, `sessionId`, unchanged from
    the v0.18.2 finding in MUL-5770.

A version gate cannot separate those — they are different projects, not
different releases. The catalog can, and does: an empty catalog means no
picker and no set_config_option call, so admitting `hermes` to
acpCatalogThinkingProviders is safe for both. This is the case the
catalog-driven design was built for.

Also corrects the docs that stated hermes has no reasoning control. That
is now binary-specific rather than universally true, and Copilot takes
over as the example of a runtime with no effort dial at all.

Co-authored-by: multica-agent <[email protected]>

* test(handler): retarget the no-reasoning-control fixture from hermes to copilot

These tests used hermes as the canonical runtime with no effort dial, so
enabling it broke them in CI — they passed locally only because the
handler suite skips without a database.

Copilot takes over the role: it discovers over ACP but executes through
its own CLI, so there is genuinely no live ACP session to carry an
effort. The behavior being pinned is unchanged — a capability gap must
not be reported as an unrecognised token.

Adds the inverse assertion in the same test: hermes now accepts a level
at the provider gate, since jcode applies one. Whether a picker appears
is settled per session by the discovered catalog, not here.

Verified against a real Postgres this time rather than the skip path.

Co-authored-by: multica-agent <[email protected]>

* fix(agent): keep the capability 400 for hermes runtimes with no effort dial

Opening the thinking_level gate for the hermes provider regressed the
majority of that provider's users. Hermes Agent, not jcode, is what most
people run there, and it advertises no effort option — so a CLI
`--thinking-level high` went from an immediate, accurate 400 to a 201
followed by the daemon silently dropping the level at task time.

The provider name cannot distinguish the two binaries, but the model
catalog the daemon already reports can: a hermes runtime whose catalog
advertises no thinking levels is a Hermes Agent install. Consult it in
the create and update paths and restore the capability message for that
case, leaving jcode unaffected.

Fails open on purpose. A cold cache, an offline daemon, or a
never-discovered runtime all skip the check, so a valid jcode agent is
never blocked before its first discovery; the daemon's pre-execution
guard remains the backstop.

Verified against a real Postgres, including the fail-open paths.

Co-authored-by: multica-agent <[email protected]>

* fix(agent): refuse a thinking level on an undiscovered hermes runtime

Review caught that the fail-open branch was not the transient cold start I
assumed. The model catalog is written only by ReportModelListResult, i.e.
only after a client explicitly asks for a model list, so a caller who works
through the CLI and never opens a model picker stays undiscovered
indefinitely. Reading "unknown" as "supported" therefore left the majority
Hermes Agent case exactly where it started: `--thinking-level high` accepted
and persisted, then silently dropped by the daemon at task time.

Replace the boolean with three states — present / absent / unknown — and
refuse on unknown, but only for providers whose name does not determine the
binary. `hermes` is the sole member; reasonix names one binary that does
support an effort, so an undiscovered reasonix runtime stays allowed rather
than being blocked before its first discovery.

Unknown gets its own message rather than reusing "does not support a
per-agent reasoning effort". That sentence would be actively false for a
jcode runtime, sending its owner to look for a limitation that isn't there;
naming the missing evidence points at what actually resolves it.

Also adds the execution-chain coverage the review asked for: jcode-shaped
ACP stubs prove set_config_option(reasoning_effort) goes out before
session/prompt on both session/new and session/resume, and a Hermes
Agent-shaped payload produces no config call at all. That is the substitute
evidence for having no real jcode to test against.

Skill and source map updated so the documented guarantee matches the code
rather than being stronger than it.

Co-authored-by: multica-agent <[email protected]>

* fix(agent): word the undiscovered-runtime 400 for create as well as update

The message pointed at "the agent's model picker", which does not exist on
the create path — the agent is not created yet. "the model picker for this
runtime" is accurate on both paths (review nit, PR #6814).

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Bohan-J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-13 16:42:47 +08:00
2c74143096 fix(agent): preserve effort for Claude context variants (#6748)
Co-authored-by: Eve <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-11 15:08:44 +08:00
Chan Nyein Thaw f97db4fc5a fix(agent): support Pi thinking levels through RPC model discovery (MUL-5924) (#6629)
Discover Pi models through an ephemeral RPC session so per-model thinking levels come from Pi's own reasoning/thinkingLevelMap data, and pass the selected level through as --thinking.

Splits the 15s discovery budget (7s RPC / 8s table) and matches RPC responses on the echoed command as well as id, so an RPC surface that accepts the mode but never answers cannot starve the --list-models fallback.

Closes #6628
2026-08-08 23:53:18 +08:00
ca2541a2ae fix(agents): explain why a runtime refuses thinking_level (MUL-5770) (#6447)
Setting a reasoning effort on a Hermes agent failed with "thinking_level
\"high\" is not a recognised value for runtime \"hermes\"", which reads
like a typo. It is not: Hermes has no reasoning control on the surface
Multica drives it over, so no spelling of the value can ever work.

Add ThinkingControlSupported as the capability predicate behind the
existing token gate, and use it so the API answers with the capability
gap instead of blaming the value. The Hermes evidence (ACP session/new
advertises no configOptions, set_config_option is inert, _make_agent
never sets reasoning_config) is recorded next to the predicate so the
next reader does not re-derive it, with a pointer from hermes.go.

No behaviour change to which values are accepted.

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-08-05 21:13:52 +08:00
6992c58de3 MUL-5185: add Codex Fast mode (#5821)
* feat(agents): add Codex fast mode (MUL-5185)

Co-authored-by: multica-agent <[email protected]>

* fix(agents): make Codex Fast override authoritative

Co-authored-by: multica-agent <[email protected]>

* fix(agents): remove Codex Fast config conflicts

Co-authored-by: multica-agent <[email protected]>

* chore: refresh checks after conflict resolution

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: Eve <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-07-23 15:40:18 +08:00
6b980a8e71 feat(models): add Codex gpt-5.6 series (sol/terra/luna) to model list & pricing (MUL-4347) (#5188)
* feat(models): add Codex gpt-5.6 series (sol/terra/luna) to model list & pricing (MUL-4347)

Co-authored-by: multica-agent <[email protected]>

* fix(models): official gpt-5.6 pricing, exact aliases, max/ultra effort levels (MUL-4347)

- Replace provisional gpt-5.6 rates with OpenAI's official announcement
  values (sol 5/30, terra 2.5/15, luna 1/6); cache read 0.1x input, cache
  write 1.25x input (frontend + backend, kept in sync).
- Anchor gpt-5.6 price aliases to exact match so unknown suffixed variants
  surface as unmapped instead of borrowing a tier.
- Add Codex 0.144.1 max/ultra effort levels to the label map and server
  enum so the daemon-advertised catalog matches what the API can persist;
  add a catalog->API contract test.
- Clarify that the codex Default flag is the effort-validation anchor, not a
  user-facing badge.
- Note the cache-write measurement limitation (codex usage stream doesn't
  report cache-write tokens yet).

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: J <[email protected]>
Co-authored-by: multica-agent <[email protected]>
2026-07-10 13:00:34 +08:00
Jiayuan Zhang eb6dffdbc6 MUL-3341: clear incompatible model on runtime switch
Closes MUL-3341
2026-06-17 08:23:20 +02:00
Bohan Jiangandmultica-agent 2bec2221d2 feat(agent): per-agent thinking_level for claude + codex (MUL-2339) (#2865)
* feat(agent): persist thinking_level per agent (MUL-2339)

Adds a nullable `thinking_level` column to the `agent` table so the
backend can route a runtime-native reasoning/effort token (e.g. Claude's
`xhigh`, Codex's `minimal`) through to the agent CLI on every dispatch.

The column is intentionally TEXT rather than an enum — Claude and Codex
publish overlapping but distinct vocabularies and we want the persisted
value to round-trip exactly through whichever CLI receives it. NULL is
the "use runtime default" sentinel that every downstream consumer reads
as "do not inject --effort / reasoning_effort".

This commit is just the storage layer (migration + sqlc); subsequent
commits wire it through the API, daemon, and agent backends.

Co-authored-by: multica-agent <[email protected]>

* feat(agent-backend): inject reasoning effort for claude + codex (MUL-2339)

Extends ExecOptions with a runtime-native ThinkingLevel string and wires
it into the Claude and Codex backends. Discovery is driven by the local
CLI so the daemon advertises whatever the host install supports rather
than a hand-maintained list that goes stale.

Per Elon's PR1 review:
- Claude: parses `claude --help` to learn the `--effort` superset and
  projects through a per-model allow-list (xhigh is Opus-only; max is
  session-only on the smaller models). Falls back to a conservative
  static list when the binary is missing or help drift hides the line.
- Codex: drives `codex debug models --output json` so per-model
  reasoning subsets and the documented default come directly from the
  CLI. The older config-error probe trick is gone — the JSON path is
  stable and doesn't pollute stderr with an intentional misconfig.
- Cache key includes (provider, executablePath, cliVersion) so a CLI
  upgrade invalidates entries that referenced the older help / catalog.

Per Trump's PR1 constraint, all three Codex injection points
(thread/start.config, thread/resume.config, turn/start.effort) flow
through one helper (`applyCodexReasoningEffort`) so they cannot drift
independently. The shared `codexReasoningCases` fixture in
`thinking_test.go` asserts the same value→{shape, key} contract at
each site for every level the runtimes know about.

Claude's `--effort` is also added to `claudeBlockedArgs` so a user
custom_args entry can't silently outvote the daemon-injected value.

Co-authored-by: multica-agent <[email protected]>

* feat(api): wire thinking_level through API + daemon contract (MUL-2339)

End-to-end plumbing for the per-agent reasoning/effort setting:

- AgentResponse / TaskAgentData now carry `thinking_level`; the daemon's
  claim response includes it and the daemon's executor passes it through
  to agent.ExecOptions, where the Claude and Codex backends already know
  what to do with it.
- ModelEntry on the runtime-models wire format gains a `thinking` block
  carrying `supported_levels` + `default_level` per model so the UI can
  render a runtime-aware picker without the server having to know about
  the local CLI install. `handleModelList` projects the agent-package
  catalog (including the new Thinking field) into the wire shape.
- CreateAgent / UpdateAgent gate the field with a synchronous provider
  enum check (claude / codex only today). UpdateAgent is tri-state:
  field omitted = no change, "" = explicit clear (new
  `ClearAgentThinkingLevel` query, mirrors the existing mcp_config null
  pattern), non-empty = validate then set.

Per Trump's PR1 review, the API NEVER auto-clears on a runtime/model
swap and ALWAYS returns 400 on an unknown literal value — same shape
across CreateAgent, UpdateAgent, and combined patches that move
runtime + level in one request. Per-model combination failures (e.g.
`xhigh` against a model that only supports up to `high`) surface as a
daemon-side task error, not a silent server-side rewrite.

TS types follow the same shape: `Agent.thinking_level`,
`CreateAgentRequest`/`UpdateAgentRequest` add the field, `RuntimeModel`
grows a `thinking` block. Older backends omit the field, which the
front-end treats as "no picker for this model" — installed desktop
builds keep working.

Co-authored-by: multica-agent <[email protected]>

* fix(agent): correct codex debug models argv + pin via runner test (MUL-2339)

`codex debug models --output json` is rejected by codex-cli 0.131.0 —
the subcommand emits JSON on stdout by default and has no `--output`
flag. Drop the flag and add `--bundled` to skip the network refresh
discovery doesn't need. Move the argv to a package-level var and add
a test that runs a fake `codex` to assert the binary actually
receives exactly `debug models --bundled`, so the contract can't
silently drift on the next refactor.

Also teach ValidateThinkingLevel to resolve an empty model to the
provider's default model entry. Without this, every default-model
task with a persisted thinking_level would be misjudged "unknown
model" by the daemon guard.

Co-authored-by: multica-agent <[email protected]>

* fix(api): reject runtime switch that would leave invalid thinking_level (MUL-2339)

A PATCH that changed `runtime_id` without touching `thinking_level`
used to silently keep the existing value, so a Claude agent storing
`max` could land on a Codex runtime where `max` is not a recognised
token at all, and the daemon would receive a literal-invalid level.

Hold the same "always 400 on literal-invalid, never silent coerce"
rule on this implicit path. When runtime_id changes and the existing
value is not in the new provider's enum, return 400 with the
recovery options (clear via `thinking_level=""` or re-set in the
same PATCH).

Add coverage for both the kept-when-still-valid and the rejected
cases, plus the two recovery paths (clear and replace).

Co-authored-by: multica-agent <[email protected]>

* fix(daemon): guard runTask with per-model thinking_level validator (MUL-2339)

ValidateThinkingLevel existed but had no call site — `task.Agent.
ThinkingLevel` flowed straight into ExecOptions, so `xhigh` configured
on a non-Opus Claude model, or API-side stale values that escaped the
provider enum gate, would be injected anyway.

Run the validator before building ExecOptions. Invalid combinations
log a warning and drop the level instead of failing the task: the
agent still runs, just at the runtime's default reasoning effort.
Discovery errors fail open (keep the level, let the CLI surface any
objection) so a transient `claude --help` failure can't strand work.

Empty model is forwarded as-is; the validator resolves it to the
provider's default model internally per the cross-package contract.

Co-authored-by: multica-agent <[email protected]>

* chore(agent): drop stale `--output json` comments + unused scanner (MUL-2339)

Codex CLI's `debug models` subcommand emits JSON without an `--output`
flag, and `parseCodexDebugModels` never read from the bufio.Scanner.
Sync the comments with the actual invocation and remove the dead init.

Co-authored-by: multica-agent <[email protected]>

---------

Co-authored-by: multica-agent <[email protected]>
2026-05-20 12:30:10 +08:00