Preserve Codex and XAI upstream message-too-big closes across the Responses WebSocket boundary while treating them as request-scoped errors. Prevent CPA reconnect and credential cooldown/fallback, scope close state to the active connection, retain terminal errors under backpressure, and ensure downstream disconnect cleanup cannot wait behind a blocked writer.
- Implemented logic to differentiate between generic and explicit model-not-found errors for the `count_tokens` endpoint.
- Introduced `recordAvailabilityNeutralResult` to ensure models remain active for generic endpoint errors.
- Expanded test coverage to verify behavior for various 404 scenarios, including error propagation and model suspension handling.
- Refactored error-parsing utilities to identify structured and nested model-not-found errors.
Closes: #4410
- Introduced CPA trace ID middleware to inject trace IDs in response headers.
- Added support for CPA trace ID generation and validation.
- Updated metadata handling to include stable auth indices and trace ID callbacks for enhanced request tracking.
- Enhanced tests to verify CPA trace ID propagation and reserved header handling across middleware, metadata, and handlers.
Closes: #4370
- Enhanced response handling to support `response.incomplete` events across translator components and executors.
- Refactored `resultErrorFromError` to centralize error conversion and consolidate repeated logic for constructing custom errors.
- Updated handling of `max_output_tokens` and similar terminal reasons in OpenAI and Gemini translator workflows.
- Introduced `IsRequestScoped` and `IsRequestScopedError` to differentiate between request-scoped and non-request-scoped errors.
- Added comprehensive test cases for incomplete responses, terminal failures, and error propagation in both streaming and non-streaming conditions.
Closes: #3055
Propagate client generate metadata through usage records while keeping
legacy callers that omit the field enabled by default. Preserve an
existing context generate=false when metadata does not set the value,
and normalize omission to true only at publishing boundaries.
- Implemented `SelectAuthByKind` to filter and select credentials by kind with retry logic and error handling.
- Updated Codex Alpha Search to enforce OAuth credentials and sanitize upstream request body.
- Added unit tests to validate `SelectAuthByKind` behavior across scenarios.
- Introduced `sanitizeCodexAlphaSearchBody` for removing sensitive fields from API payloads.
Closes#4289
- Implemented `homeForceMappingAliasResult` for handling forced alias mapping in the authentication process.
- Updated `resolveExecutionAliasResultForRequested` to incorporate home force mapping.
- Introduced new attributes (`home_force_mapping` and `home_original_alias`) for managing alias behavior.
- Extended `homeAuthDispatchResponse` with fields for force mapping and original alias.
- Added test cases to validate force mapping logic and edge cases.
* fix(usage): preserve request service tier metadata
* fix(usage): collapse request tier metadata
* fix(usage): emit only service_tier and response_service_tier
Collapse request tier to service_tier for usage records and queue payloads. Keep RequestServiceTier as a deprecated input-only alias for older plugin callers.
- Introduced `xai-api-key` to configuration for supporting xAI provider integration.
- Added API key operations: creation, updates, deletion, and sanitization.
- Implemented backend enhancements for credential-specific routing, response filtering with model aliasing, and WebSocket handling for xAI services.
- Included extensive test coverage to validate configuration parsing, CRUD operations, and the new functionalities.
- Updated related modules to handle `xai-api-key`, including the watcher, management handlers, and configuration sanitizers.
- Added `tryRefreshAfterUnauthorized` to refresh OAuth credentials on 401 errors during requests.
- Implemented `refreshLocks` to prevent concurrent refreshes for the same auth ID.
- Updated auth/state handling to reset unauthorized model states and resume operations after a successful refresh.
- Enhanced refresh logic with error handling, synchronization, and state updates.
Closes: #4087
- Added fallback and suspension handling for `invalid_grant` errors in auth conductor.
- Introduced helper methods to detect `invalid_grant` error patterns.
- Updated tests to validate `invalid_grant` fallback and suspension scenarios in execution and stream flows.
Closes: #4120
- Introduced API handlers and executor logic for Google Interactions
- Added request and response transformations for OpenAI and Claude Interactions.
- Integrated Gemini API with Interactions support.
- Updated tests to validate Interactions request parsing and error handling.
- Refactored translator logic for Interactions data flows.
shouldRetryAfterError approves a retry using the pre-jitter wait, so the jitter added in waitForCooldown could push the actual sleep up to 2s past the configured max-retry-interval ceiling. Clamp the jitter range to the remaining headroom below maxWait so the documented maximum stays a hard bound; waits at the ceiling simply sleep unjittered.
Previously every 429 incremented the backoff level unconditionally, so N concurrent in-flight failures from a single quota event inflated the level by N at once (12+ failures jump straight to the 30 minute plateau even when the upstream limit clears in seconds). Failures that land while a previous quota window is still open now reuse that window instead of escalating, so the ladder advances at most once per window. Provider supplied retry hints still take effect unchanged.
waitForCooldown also slept until the exact recovery deadline, waking every waiting request in lockstep against the single recovered credential and re-escalating it in one burst. Cooldown waits now carry a small random jitter (up to wait/4, capped at 2s) to spread the wakeups.
Force-mapping rewrites streaming OpenAI Responses SSE through StreamRewriter before the /v1/responses websocket forwarder. Antigravity/Gemini and Codex emit frames without reliable trailing newlines, so buffered chunks glued as ...}event:..., ...}data:..., or event:/data: without separators. The rewriter dropped pending tails and downstream WS synthesized 408 without response.completed even when upstream returned HTTP 200.
- safeReplaceGlued for }event: and }data: when data JSON is complete\n- Finish flush with glue normalization and line-wise fallback\n- Newline between pending event: and next data: line (Codex scanner lines)\n- Regression tests: Antigravity sim, Codex data lines, force-map WS forward
* fix(oauth): force-map responses to config alias not request suffix
Rewrite client-visible model fields using oauth-model-alias Alias from config
(e.g. gpt-5.4-fast) instead of mirroring the requested model string when the
client adds thinking suffixes (e.g. gpt-5.4-fast(high)). Upstream routing
still preserves suffixes on the resolved upstream model.
* test(oauth): gate response rewrite on ForceMapping and document OriginalAlias
- Document that OriginalAlias is only applied when ForceMapping is true
- Set OriginalAlias to config alias only when force-mapping is enabled
- Add tests: no non-stream/stream payload rewrite when ForceMapping is false
* test(oauth): assert disabled force mapping response
Add OAuthModelAlias.ForceMapping to rewrite client-visible model fields in non-stream and SSE stream responses while leaving request-log upstream payloads unchanged.
- Wire config, watcher diff, and conductor execute/stream paths
- Rewrite force-mapped responses in Antigravity credits fallback paths
- Prevent buffered SSE chunks from being emitted by line-wise fallback
- Preserve thinking suffixes for same-base force-mapping aliases
- Cover codex, antigravity, kimi, and xai with live-derived fixtures and regression tests
- Document a conservative antigravity-only force-mapping example in config.example.yaml
- Introduced `AuthKind` and `AuthSourceKind` methods for improved credential type and source classification.
- Replaced legacy fallback logic with normalized attribute-based handling in `AuthKind`.
- Consolidated metadata inspection for both API key and OAuth attributes.
- Updated calls to replace direct `AccountInfo` usage with `AuthKind` and `AuthSourceKind`.
- Enhanced unit tests to validate explicit and fallback credential resolution scenarios.
- Added `RefreshAPIKeyModelAlias` for explicit alias table rebuilds.
- Introduced deferred rebuild support with `WithDeferredAPIKeyModelAliasRebuild` and context flag validation.
- Implemented `openAICompatibilityRegistrationCache` to streamline OpenAI compatibility model registrations.
- Updated executor and model registration workflows to utilize cached compatibility data, improving efficiency in batch operations.
- Adjusted max worker limits dynamically based on model categories.
Closes: #3953
- Added logic to expand single auth JSON payloads into multiple plugin virtual auth records.
- Updated related API endpoints such as `PatchAuthFileStatus` and `DeleteAuthFile` to handle plugin virtual auths with rollback mechanisms.
- Introduced `NormalizePluginOAuthCallbackProvider` and other normalization functions for better handling of OAuth callbacks.
- Enhanced tests to validate multi-auth parsing, rollback behavior, and API response consistency.
- Introduced a new `/reset-quota` API endpoint in the management handler to clear quota and cooldown state for auth records.
- Implemented `ResetQuota` method in the auth manager to handle runtime and registry state resets for affected models.
- Added tests to validate quota reset behavior, including proper state updates and registry consistency.
- Refactored utility functions to support deduplication and registered models handling in quota resets.
Closes: #3866
- Introduced `SetOAuthModelAliasesAttribute` and `OAuthModelAliasesFromAttributes` for managing per-auth model aliases.
- Enhanced OAuth model resolution logic to prioritize per-auth aliases over global aliases.
- Updated metadata handling to extract and sanitize per-account model aliases.
- Added tests to validate alias precedence, empty attributes, and conflict scenarios.
Closes: #3764
- Replaced direct `strings.ToLower` usage with `util.OpenAICompatibleProviderKey` for generating provider keys.
- Updated auth and executor workflows to use namespaced keys for OpenAI-compatible providers.
- Adjusted tests to validate namespaced key handling, including new test cases for provider registration and execution logic.
- Added `OpenAICompatibleProviderKey` helper in `util` for consistent key transformations.
Closes: #3600
- Introduced `CooldownStateStore` interface for managing independent cooldown state persistence.
- Implemented `FileCooldownStateStore` for storing cooldown states as per-auth `.cds` files with atomic writes and stale file cleanup.
- Enhanced `Manager` to support restoring state from `CooldownStateStore` and persisting state changes during auth updates.
- Updated tests to validate cooldown state saving, loading, concurrency handling, and error scenarios.
Closes: #3368
- Introduced `SetTransientErrorCooldownSeconds` to enable configurable cooldowns for transient errors (e.g., 408/500/502/503/504).
- Updated retry scheduling logic to use the new `nextTransientErrorRetryAfter` function.
- Modified config parsing to include `transient-error-cooldown-seconds` with support for disabling or defaulting to legacy behavior.
- Expanded tests to validate transient cooldown logic with various configurations and edge cases.
Closes: #3315
- Updated error handling in `RPopAuth` to distinguish `auth_not_found` from transport errors.
- Added a new test, `TestPickNextViaHomeClassifiesTransportErrorsAsHomeUnavailable`, to validate correct error classification and retryable property.
- Deleted `geminicli` provider and related `Apply` logic.
- Removed all translator packages specific to Gemini CLI (Claude, Codex integrations).
- Purged associated test files for Gemini CLI translation.
- Removed `GeminiAuthenticator` and all associated authentication logic (OAuth flows, token handling, refresh logic).
- Deleted internal/executor Gemini OAuth support, including bearer token handling and runtime API logic.
- Purged all tests, configs, and command-line flags specific to Gemini OAuth flows.
- Updated documentation and aliases to reflect Gemini removal.
- Renamed `parseRetryDelay` to `ParseRetryDelay` and `deleteJSONField` to `DeleteJSONField`.
- Updated references in `antigravity_executor` and tests to use the new `helps` package.
- Adjusted import paths and test cases to ensure compatibility with the new location.
- Updated README files to reflect changes in the retry logic references.
- Updated `.github/ISSUE_TEMPLATE/bug_report.md` to remove deprecated Gemini CLI mention.
* feat(plugin): add ModelRouter before auth with single-slot routing targets
## Motivation
Plugins that need to change execution based on the **original inbound request**
(protocol format, raw body, headers, query, stream flag, metadata, etc.) often
resorted to virtual/trampoline models or routing inside interceptors. This
commit adds **ModelRouter**: a pluggable layer **before** model-to-provider
resolution and AuthManager credential selection, so plugins can declare who
executes a request without spoofing the client model name.
This is a **new capability**, not a bugfix on the existing chain. With no
ModelRouter plugins loaded, behavior matches upstream.
## Pipeline placement
- `execute`, `stream`, and `count` (and image paths via AuthManager) call
`applyModelRouter()` before building `coreexecutor.Request`.
- Routing runs **before** the request interceptor (before auth), so routers see
the client’s original context. After a plugin executor is chosen, the existing
**after-auth interceptor → response/stream interceptor** chain still applies.
- Internal `ExecuteModel` / `ExecuteModelStream` (host callbacks) support
`SkipRouterPluginID` so nested calls do not re-enter the same router.
## Routing API (single slot, mutually exclusive)
`ModelRouteResponse` uses **one target slot** to avoid ambiguity when both
`TargetExecutorPluginID` and `TargetProvider` were set and the host ignored one:
| Field | Meaning |
|-------|---------|
| `Handled` | `false`: this router declines; try the next router or default path |
| `TargetKind` | `self` \| `executor` \| `provider` (pick one) |
| `Target` | `self`/`executor`: plugin ID; `provider`: built-in provider key |
| `TargetModel` | Optional on `provider` only; empty keeps client `RequestedModel` |
| `Reason` | Optional diagnostic text |
- **self**: the router plugin’s own executor (`Target` normalized to the router’s plugin ID).
- **executor**: another plugin’s executor; host pre-checks with `executorPluginReady()`
(executor declared and provider identifier resolvable) to avoid handled routes that 500 at execution.
- **provider**: skip registry model resolution; fixed built-in AuthManager path; optional
`TargetModel` for execution model only—**does not** change outward requested-model metadata.
Routers run in **descending plugin priority** (tie-break: ascending plugin ID). Panic, error,
invalid target, or unavailable executor/provider → log and **fall through to the next router**;
if none handle, use the original provider+auth flow.
## Context exposed to routers
`ModelRouteRequest` includes:
- `SourceFormat`, `RequestedModel`, `Stream`
- `Headers`, `Query`, `Body` (defensive copies)
- `Metadata` (best-effort read-only context snapshot)
- `AvailableProviders`: built-in provider keys with at least one **non-disabled** auth
(`AuthManager.AvailableProviders()`). **Does not** reflect per-model cooldown or transient
unavailability—treat as an optimistic snapshot.
Adds `AuthManager.HasProviderAuth()` and `AvailableProviders()`, excluding `Disabled` and
`StatusDisabled` auths consistently with credential selection.
## Host and RPC
- Go plugins: `pluginapi.ModelRouter` + `RouteModel()`.
- RPC plugins: `pluginabi.MethodModelRoute` (`model.route`), capability flag `model_router`.
- `pluginhost.Host` implements `RouteModel` / `RouteModelExcept`; handlers use
`SetModelRouterHost` or a `PluginHost` type assertion; **direct executor** paths use
`ExecutePluginExecutor*` / `CountPluginExecutor`.
- No bundled example ModelRouter plugin; capability is active only when a third-party plugin
declares `model_router` and loads.
## Plugin RPC schema (policy A, upstream-aligned)
- `pluginabi.SchemaVersion` stays **1**: capability additions (`model_router`, `model.route`)
do not bump the number; increment only on breaking RPC JSON changes.
- Host sends `schema_version` at register; reject only if the plugin declares a **higher**
version than the host.
- No unpublished “ModelRouter requires schema ≥ 3” gate (v3 single-slot API was never public).
- Existing plugins and examples without `model_router` (`schema_version: 1`) need no changes.
- RPC ModelRouter: `schema_version: 1` + `model_router: true` + implement `model.route`.
## Path consistency within this commit
- Provider routes reuse image-only model checks (e.g. `gpt-image-2`) on the normalized model,
same as the default AuthManager path.
- `count` aligned with execute/stream: `SkipRouterPluginID`, query/headers injection,
interceptor skip semantics.
- Handlers: `modelRoutersEnabled` treats hosts without `HasModelRouters` as disabled
(same as before ModelRouter existed); `pluginhost.Host` implements the detector.
- API docs: `ModelRouter` explicitly includes built-in **provider** targets (in addition to
plugin executors and the router’s own executor).
## Testing
go test ./internal/pluginhost ./sdk/api/handlers ./sdk/pluginapi ./sdk/pluginabi ./sdk/cliproxy/auth
go build -o test-output ./cmd/server && rm test-output
go test ./...
* fix(handlers): address ModelRouter review feedback
- Use modelExecutionQuery for plugin executor and AuthManager paths so
inbound URL query matches router/header behavior
- Guard queryFromContext when gin Request.URL is nil
- Read plugin executor stream chunks via nextStreamChunk to exit on cancel
- Drop redundant clonePluginMetadata on capability record meta
Tests cover query propagation, stream cancel, and nil URL safety.
* feat(plugin): add Claude web search router example
Add a Claude Code web_search ModelRouter example that can route matching Claude requests through Antigravity, Codex, xAI, or Tavily.
The plugin includes executor orchestration, backend fallback/penalty handling, Tavily API key support, Claude-compatible response assembly, stream forwarding, and focused unit coverage for detection, fallback routing, model resolution, penalties, stream forwarding, and Tavily behavior.
Verification: go test -count=1 ./... in examples/plugin/claude-web-search-router/go; go build -buildmode=c-shared for the plugin; go build ./cmd/server; live local CPA curl coverage for plugin load, four explicit routes, fallback, and Codex spark routing.
* fix(pluginhost): validate executor routes before fallback
* fix(pluginhost): skip oauth-only executor routes
- Implemented helper methods `IsConfigAPIKeyAuth` and `toggleConfigAPIKeyExcludedAll` for managing config API key exclusions.
- Updated API request handling to support enabling/disabling config API key exclusion patterns.
- Added test coverage to validate exclusion toggling logic and persistence behavior.
- Refactored duplicate code for identifying config API key auth entries into reusable utilities.
- Developed `XAIWebsocketsExecutor` for handling xAI Responses via WebSocket transport.
- Introduced session and state management with `codexWebsocketSessionStore` and `xaiWebsocketIDStateStore`.
- Added robust ID mapping for upstream and downstream request/response sequences.
- Enhanced error propagation and handling of WebSocket terminal events.
- Included utility methods for WebSocket request preparation, connection management, and state tracking.
- Added foundational support for compact and streamed responses via enhanced session tracking.
- Updated Antigravity Credits fallback to handle KV store unavailability as a service error.
- Enhanced signature caching mechanisms with request-time KV access and sliding expiration.
- Added and improved tests for KV client interactions, including error handling and expiration behaviors.
- Introduced `CacheSignatureBestEffort` for non-critical signature caching and clarified function flows with required context.
- Ensured consistent error reporting for missing or unavailable KV stores in various scenarios.
- Replaced direct `homekv` calls with injectable KV client interfaces for `antigravity` and `codex_reasoning_replay` modules.
- Improved error reporting and handling for KV operations, including `KVGet`, `KVSet`, `KVDel`, and `KVExpire`.
- Introduced dedicated fake KV clients for expanded and granular test coverage.
- Added new unit tests to validate KV client behaviors and error scenarios, ensuring robustness and sliding expiration functionality.
- Introduced `applyRequestAfterAuthInterceptor` to modify requests after credential selection and before executor translation.
- Added `InterceptRequestAfterAuth` method across plugin adapters with corresponding tests for context validation.
- Enhanced format resolution logic (`requestToFormat`) to support additional providers and formats.
- Updated JavaScript handler to include a new `on_after_auth_request` hook for post-auth request handling.
- Refactored interceptor methods for clarity and better encapsulation of request/response lifecycles.
- Removed `pluginDelegateRoundRobin` and related logic to streamline plugin scheduler management.
- Consolidated scheduler strategies under `builtinSchedulerStrategy` with `pickViaBuiltinScheduler`.
- Introduced new methods `pickSingleWithStrategy` and `pickMixedWithStrategy` for strategy-specific behavior.
- Updated tests to reflect changes, including added coverage for round-robin and mixed-provider scenarios.
- Improved maintainability by unifying scheduling logic and reducing redundant structures.
- Added `pluginSchedulerState` interface with `HasScheduler` method for improved plugin scheduler state checks.
- Updated `Manager.hasPluginScheduler` to handle `HasScheduler` logic.
- Implemented and tested fast-path handling for inactive plugin schedulers, including mixed provider scenarios.
- Expanded unit test coverage to ensure correct behavior in various scheduler states.
- Added a Go scheduler plugin demonstrating CLIProxyAPI capabilities, such as `plugin.register`, `plugin.reconfigure`, and `scheduler.pick`.
- Implemented methods for plugin configuration, built-in scheduler delegation (`fill-first`, `round-robin`), dynamic candidate selection, and error handling.
- Extended `pluginhost` with scheduler handling, candidate normalization, and fallback mechanisms.
- Included examples, tests, and detailed documentation for scheduler usage and implementation.
- Implemented command-line flag registration and execution for plugins with priority-based conflict resolution.
- Enabled plugin-owned command-line flag execution and persistence of plugin-auth data.
- Added new `Host` methods to support command-line capabilities, including flag normalization, validation, and execution state management.
- Introduced unit tests to ensure coverage for command-line plugin functionality, including auth data persistence.
- Updated configs to normalize plugins during initialization.
- Introduced `publishErrorEvent` in `Manager` to publish error events to Redis.
- Implemented error event structure to capture authentication errors with detailed metadata.
- Added test cases for error event publishing, subscription, and Redis protocol handling.
- Enhanced error and usage queue handling with `SubscribeErrors` and `EnqueueError`.
Closes: #3701
- Introduced `Manager.Remove` to delete runtime auth and unschedule associated tasks.
- Updated handler logic to directly remove auth instead of marking as disabled.
- Added tests to validate removal, unscheduling, and runtime state handling.
- Added a test to validate `skipPersist` behavior during registration.
- Enhanced `Remove` test to verify auto-refresh loop state before and after removal.
Closes: #3690
Introduce Cloudflare challenge detection for 403 errors in the Auth
Manager. Apply a progressive rate-limiting cooldown ladder using the
existing BackoffLevel field instead of a hard 30-minute credentials
suspension. This ensures challenged requests fall through to subsequent
credentials and recover exponentially.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Parse Home refresh auth envelopes so refreshed access tokens are used instead of returning missing access token.
Stop retrying when Home dispatch returns an auth that already failed within the same request.
hkfires