CaIon
9df450fe54
Plugin polling hooks previously ran against a hollow context: parseTaskResult
and parseBatchResult received {} / nil, buildQueryRequest received a
{task_id, action} map under the misleading name requestBody, and batch hooks
saw only bare task ids. The per-task poller also never looked at the upstream
HTTP status, and every built-in plugin papered over unrecognized bodies with
`|| "IN_PROGRESS"`, so a 404, a revoked key, or a shape the plugin did not
know would sit in IN_PROGRESS for the full 24h TASK_TIMEOUT_MINUTES while
holding the user's pre-charged quota.
Contract (docs/plugin-api v1.d.ts, v1.md, v1.schema.json):
- TaskQueryContext is declared separately from DriverContext and rebuilt from
the persisted Task row: taskId, publicTaskId, action, model, upstreamModel,
baseUrl, apiKey, authHeader, auth, data, state. Query-side requestBody is
removed; the original request is not persisted and hooks that need a
request-derived value must save it into state at submit time.
- parseTaskResult / parseBatchResult receive a third {status, headers}
argument. Batch hooks receive tasks[] with one TaskQueryContext per task.
- NormalizedTaskResult accepts status "UNKNOWN" meaning "I do not recognize
this body". Falling back to IN_PROGRESS for unknown shapes is forbidden;
`plugin lint` warns on the literal.
- parseSubmitResponse / parseTaskResult / parseBatchResult may return `state`.
Task.Data remains a per-round snapshot overwritten on every valid parse;
state is plugin-owned, persisted in TaskPrivateData.PluginState, preserved
when a hook omits it, byte-capped like taskData, and never exposed through
presenter views.
Host (service/task_polling.go, relay/channel/task/jsplugin/adaptor.go):
- TaskPollingAdaptor / BatchTaskPollingAdaptor take *model.Task and the
*http.Response so the adaptor can build the full context; jsplugin is the
only implementation.
- HTTP classification before the plugin sees the body: 2xx -> plugin;
404/410 -> FAILURE and refund; 401/403 -> poll failure plus a channel-scoped
warning, no auto-disable; 429/5xx/transport -> poll failure; other 4xx ->
plugin with the status visible, counted as unrecognized if the plugin still
reports a non-terminal state.
- TaskPrivateData.PollFailures counts consecutive poll failures (transient
HTTP, auth, transport, hook error, UNKNOWN). It is persisted through the
existing UpdateWithStatus CAS so a concurrent terminal transition on another
instance is never clobbered, and reset on any valid 2xx non-terminal parse.
Reaching TASK_POLL_MAX_FAILURES (default 20, <= 0 disables) fails the task
with the last classification and HTTP code in fail_reason and runs the
existing settle/refund chain exactly once. sweepTimedOutTasks and its
1440-minute default are unchanged as the outer backstop.
- Unrecognized bodies are logged at WARN with a bounded redacted copy since
Task.Data is intentionally not overwritten on that path.
Plugins (all ten bumped one patch version):
- jimeng persists the outbound req_key in state and reads it back in
buildQueryRequest, replacing dead reads of ctx.data / ctx.requestBody that
never resolved.
- sunoapi batch hooks read tasks[] instead of the removed requestBody.
- hailuo treats base_resp.status_code != 0 as FAILURE before the status table.
- kling, vidu, sora, alibaba, doubao, hailuo, jimeng return UNKNOWN with the
raw upstream status in reason on table miss.
- google and vertex-ai treat a missing `done` as in-progress: Google
long-running operations omit proto3 default fields, so a running Veo
operation has no `done` key at all. Only a body without an operation name is
UNKNOWN. plugins/veo_poll_test.go locks this so the poll-failure cutoff can
never fail a rendering Veo task.
Tests cover the classification table end to end against a real DB (404
immediate refund, 429xN refund, 401 increments without status change, 2xx
reset, UNKNOWN increments, state preserved vs replaced, PollFailures survives
the CAS write), the query-context shape, UNKNOWN on unrecognized bodies, and
the absence of PluginState/PollFailures from TaskView. Controller tests derive
the kling factory version from the embedded manifest instead of hardcoding it.
Task plugin API v1
Task plugins are single-file synchronous ECMAScript modules. The plugin contract
is currently unreleased; v1.schema.json and
v1.d.ts are the authoritative v1 contract.
Contract and lifecycle
Every plugin exports meta, buildSubmitRequest, parseSubmitResponse, and
parseTaskResult. A per_task plugin also exports buildQueryRequest; a
batch plugin exports buildBatchQueryRequest and parseBatchResult.
meta.author.name is required and meta.author.url, when present, must be an
absolute HTTP(S) URL. This is self-declared attribution; a future marketplace's
verified publisher identity is a separate host-owned record.
Plugins may declare authenticated vendor-native meta.routes and claim
host-owned names through meta.protocols. Submit and dynamic routes name a
native decoder and presenter; query routes name only a presenter. Protocol
bindings are registered once by the host registry, and protocol decoders receive
the host-parsed body union plus the pinned model. Shared protocol hooks are
synchronous transformations; Go owns connections and wire framing.
The host selects a channel, invokes the request-building hook, validates the returned URL against the channel host, performs HTTP, and gives the decoded response to the matching parse hook. It owns persistence, retries, polling, billing, and settlement. Plugins only transform data and report usage facts. See v1.d.ts for signatures and v1.schema.json for machine-readable shapes.
Plugins that expose task outputs export listArtifacts(task) and
buildContentRequest(ctx) together. Artifacts are projected on explicit reads
from persisted Task.Data; they are never stored as a second source of truth.
The list contains only stable key, type, and optional mimeType fields.
The content hook receives the selected key, raw decoded task data, the explicit
private upstream task id, the producer plugin version, channel authentication,
and a safe client Range/conditional-header subset. Its URL and headers exist
only for that proxy request.
When a Responses observation reaches persisted SUCCESS, the host also runs
the pinned plugin's listArtifacts and injects a read-only
ctx.artifacts[key] = {key, type, mimeType?, url} map into renderEvents or
renderFinal. Each url is a long-lived host-signed capability URL, never the
provider URL from Task.Data. Nonterminal and failed tasks receive no artifact
map. Capability construction or rendering failure fails only that Responses
observation; it cannot change the task, billing settlement, or refunds.
The absolute URL uses TaskPublicAddress, falling back only to
ServerAddress; multi-node deployments must share the effective
CRYPTO_SECRET.
Dashboard artifact reads return each content_url (or the legacy
legacy_content_url) directly, without a temporary URL exchange. Capability
generation and verification are stateless and have no expiry; after
verification the host still loads the task, owner, and plugin needed to serve
the artifact. Rotating CRYPTO_SECRET invalidates issued URLs. The access
query is redacted before request logging.
Deployment boundaries and concurrency environment variables are documented in
v1.md.
The host treats protocols.openai_video.render as a standard DTO, not an arbitrary
JSON passthrough. Unknown top-level fields and legacy task_id are removed,
id is forced to the public task id, and case-insensitive url entries are
removed from metadata. Provider output URLs belong only behind artifact
capabilities.
Provider-authenticated content URLs must use the channel base host or a
plugin-declared meta.allowedHosts entry. A public dynamic CDN URL may instead
set credentialless: true; the host then permits only GET/HEAD with no
plugin-supplied headers or body and applies SSRF checks to the initial URL and
every redirect.
Registry publication is generation-atomic. A request pins one plugin generation
for its full lifetime, while background polling may use a later active plugin
version. New versions must continue parsing responses for in-flight tasks.
Root administrators can inspect the local node with
GET /api/plugin/task/runtime/status. The response includes the node-local generation,
a deterministic revision of the active database overrides, the latest rebuild
outcome, and plugin-level compile or routing errors. Generation numbers are
local to a node; compare database revisions when diagnosing rollout lag between
nodes. If the database snapshot is temporarily unavailable, the endpoint keeps
serving node-local state and the last known revision with database_error set.
For live diagnosis, start the process with DEBUG=true and filter logs on
task_plugin. Plugin registry, routing, endpoint ownership, channel selection,
submit durability, polling adapters, and protocol observation emit safe
key/value lifecycle events. Request-context events carry the request id;
scheduled, background, and context-less work is labeled SYSTEM. Plugin
console.log output is also forwarded in DEBUG mode. Hook-time output is
prefixed with plugin key/version; module-initialization output may have an empty
identity during initial upload validation. Do not print credentials, headers,
request bodies, upstream payloads, or private URLs from plugin code; free-form
console output cannot be redacted by the host.
Fixtures and dry runs
A fixture case is {name?, hook, member?, args, expected?, expectedError?}.
Keep deterministic cases for every exported hook, its main error branch, batch
behavior, renderers, usage, and content requests. Run a fixture locally with:
new-api plugin lint plugin.js
new-api plugin test plugin.js --fixture golden.json
Root administrators can open the plugin detail Sandbox tab, choose a hook, and
submit an args JSON array. POST /api/plugin/task/:key/dryrun compiles the
active database source or factory source in a temporary registry and invokes
only that synchronous function. Dry runs never execute a request descriptor and
therefore never contact an upstream service.
Upload and release
Upload from the root-only task plugin page or POST /api/plugin/task with
{"source":"...","remark":"..."}. The server compiles the module, validates
v1 metadata and required exports, and rejects invalid source before saving it.
Use semantic plugin versions. Reusing a key/version with different source is
rejected; activate or roll back a stored version through the management page.
For a third-party platform, create a channel of type Task Plugin, select the
plugin key, provide an explicit base URL, and configure models. Clients may use
the plugin's declared native routes. The generic management surface remains
POST /v1/tasks/:pluginKey, GET /v1/tasks/:taskId, and
GET /v1/tasks/:taskId/artifacts plus
GET|HEAD /v1/tasks/:taskId/artifacts/:key/content.
Security boundary
Plugins have no fetch, filesystem, require, imports, async functions, or
environment access. The host limits execution time, concurrency, input size,
allowed request hosts, and resolves OAuth credentials outside JavaScript.
Multipart files enter JavaScript only as opaque references.
This is not a hard memory-isolation boundary. A plugin sees data needed for the current request and can influence an authenticated upstream request. Uploading a plugin is an administrator-level trust decision equivalent to configuring a channel credential. Review source and version diffs before activation. Never run untrusted plugins merely because they compile.
Usage hooks may return facts such as seconds, resolution, or upstream units, but must never calculate prices or attempt quota settlement. The host owns all pricing and clamps billing conversions.