mirror of
https://gh-proxy.org/https://github.com/klinsmaya/new-api.git
synced 2026-11-08 01:17:09 +08:00
feat(task-plugin): let Seedance and wan3.0 pick the video duration with -1
A plugin that declares duration-auto@1 may receive a duration of -1 (the vendor picks the length) anywhere in the request; other negative values and values above the limit are still rejected, and usage facts stay non-negative. A declaring plugin reserves the model's longest duration and settles on the usage the vendor reports. - doubao 1.3.0 declares it, forwards -1 to Ark and reserves each model's longest duration (12 s for Seedance 1.0 and 1.5 pro, 15 s for 2.0, 30 s for 2.5). A JSON duration on /v1/videos now reaches Ark. - alibaba 1.4.2 declares it, so wan3.0 smart duration also works on /v1/video/generations. Fixes #7700
This commit is contained in:
@@ -9,7 +9,7 @@ Mandatory conventions for quota, pricing, pre-consume, settlement, refund, and u
|
||||
**Billing safety invariants:** Quota/billing code MUST never produce a negative charge (a credit) from arithmetic overflow or unvalidated input. Apply defense in depth:
|
||||
|
||||
- Every user-controlled quantity that becomes a billing multiplier (image `n`, video `seconds`/`duration`, resolution/quality ratios, batch counts) MUST be bounded before it reaches quota calculation. Reject out-of-range values at request validation with a 400. Existing bounds: `dto.MaxImageN` for image generation count, `relaycommon.MaxTaskDurationSeconds` for task video duration, `maxTokensLimit` (`relay/helper/valid_request.go`) for `max_tokens`-family fields on every relay format (OpenAI, Claude, Gemini, Responses). Reuse these constants instead of introducing new ad hoc limits for the same concepts. When adding a new relay format or request DTO, bound its max-tokens and count fields in its validator from day one.
|
||||
- Watch for validation bypass paths: passthrough fields (e.g. `Extra["parameters"]`), task `metadata` maps, and multipart form fields can carry the same quantities around the standard DTO validation. Any adaptor that reads a multiplier from such a path must enforce the same bound (or clamp) locally.
|
||||
- Watch for validation bypass paths: passthrough fields (e.g. `Extra["parameters"]`), task `metadata` maps, and multipart form fields can carry the same quantities around the standard DTO validation. Any adaptor that reads a multiplier from such a path must enforce the same bound (or clamp) locally. For task plugins the host does this: it rejects out-of-range duration and count members at any depth of the decoded request. The one exception is a duration of -1 (the vendor picks the length) for plugins that declare `duration-auto@1`; they must reserve the model's longest duration and settle on the usage the vendor reports.
|
||||
- Durations parsed from media metadata are user/upstream-controlled too: audio file headers (transcription token counting, TTS response duration) and upstream deduction numbers (e.g. Kling `FinalUnitDeduction`) can claim absurd values. Convert them with saturation before they become token counts.
|
||||
- Never convert a computed quota or token count to `int` with a bare cast like `int(float64(quota) * ratio)`, `int(math.Round(...))` on unbounded input, or `int(decimal.IntPart())`. All quota rounding/conversion is centralized in `common/quota_math.go`; use those helpers: `common.QuotaFromFloat` (truncating) for float products, `common.QuotaRound` (half-away-from-zero) where rounding is intended, and `common.QuotaFromDecimal` for decimal products. `billingexpr.QuotaRound` delegates to `common.QuotaRound`. Do not reintroduce local conversion helpers or bare casts. Single-request saturation stays at the int32 boundary so batch accumulation cannot approach 64-bit wraparound; wallet/top-up conversion uses `common.WalletQuotaFromDecimalStrict` with the JavaScript-safe `common.MaxWalletQuota` boundary. Every clamp/NaN fallback is logged via `common.SysError`.
|
||||
- Saturation events are also audited: each helper has a `*Checked` variant (`common.QuotaFromFloatChecked` / `QuotaRoundChecked` / `QuotaFromDecimalChecked`) that additionally returns a `*common.QuotaClamp` when clamping occurred. Billing paths that compute a charge capture that clamp onto `relayInfo.QuotaClamp` (or thread it into task settlement) and, right before writing the consume/task log, call `attachQuotaSaturation` (in `service/log_info_generate.go`) which nests the marker under the log's `other.admin_info.quota_saturation` and emits a request-correlated `logger.LogWarn`. Nesting under `admin_info` makes it admin-only for free (non-admin log views strip `admin_info`). When adding a new billing path, use the `*Checked` variant and surface the clamp the same way so the anomaly stays auditable in both the admin log UI and backend logs.
|
||||
|
||||
Vendored
+1
-1
@@ -1,5 +1,5 @@
|
||||
export type JSONValue = null | boolean | number | string | readonly JSONValue[] | {readonly [key: string]: JSONValue};
|
||||
export type HostCapability = "json-clone@1" | "submit-sse-delta@1" | "json-order@1";
|
||||
export type HostCapability = "json-clone@1" | "submit-sse-delta@1" | "json-order@1" | "duration-auto@1";
|
||||
/** Kinds of upstream a driver can address: the vendor API itself, or another New API gateway with the same plugin installed. */
|
||||
export type UpstreamKind = "vendor" | "new_api";
|
||||
/** Host-injected on every driver hook context. With "new_api" the driver uses its own native-route prefix and the host already set Bearer credentials. */
|
||||
|
||||
@@ -253,6 +253,20 @@ driver hook call parses the ordered body again, which costs more on large
|
||||
bodies, so declare the capability only when the upstream needs it. A body
|
||||
that embeds a file placeholder is still written sorted.
|
||||
|
||||
`duration-auto@1` lets a request carry a duration of `-1`, which some vendors
|
||||
read as "the model picks the length" (Seedance on Volcengine Ark, wan3.0 on
|
||||
Bailian). Before billing, the host checks every member of the decoded request
|
||||
at any depth, including vendor parameters passed through `metadata`: a
|
||||
`duration` or `seconds` member (case, `_` and `-` are ignored, so
|
||||
`durationSeconds` counts) must be between 0 and 3600. For a plugin that
|
||||
declares the capability the host also accepts exactly `-1` there; other
|
||||
negative values and values above the limit are still rejected. Usage facts
|
||||
are unchanged and must stay non-negative, so the plugin never reports `-1` as
|
||||
a quantity. A declaring plugin reserves the longest duration the model can
|
||||
produce when it forwards `-1`, and settles the task on the usage the vendor
|
||||
reports, so the charge follows the length the model chose. A plugin that
|
||||
cannot settle on reported usage must not declare it.
|
||||
|
||||
Host functions are synchronous and must bound their own work: the JavaScript
|
||||
execution interrupt cannot preempt a running Go function. Request-scoped data
|
||||
must remain owned by the host request, not by pooled runtime globals.
|
||||
|
||||
@@ -20,7 +20,7 @@
|
||||
{"type": "object", "additionalProperties": false, "required": ["name"], "properties": {"name": {"enum": ["openai_video", "openai_image"]}, "models": {"type": "array", "minItems": 1, "uniqueItems": true, "items": {"type": "string", "minLength": 1}}}}
|
||||
]}},
|
||||
"submitResponseTypes": {"type": "array", "minItems": 1, "uniqueItems": true, "items": {"enum": ["json", "sse"]}},
|
||||
"requiredCapabilities": {"type": "array", "uniqueItems": true, "items": {"enum": ["json-clone@1", "submit-sse-delta@1", "json-order@1"]}},
|
||||
"requiredCapabilities": {"type": "array", "uniqueItems": true, "items": {"enum": ["json-clone@1", "submit-sse-delta@1", "json-order@1", "duration-auto@1"]}},
|
||||
"routes": {"type": "array", "items": {"$ref": "#/$defs/route"}}, "usageSchema": {"type": "object", "additionalProperties": {"$ref": "#/$defs/usageField"}}, "usageExamples": {"type": "array", "items": {"type": "object", "additionalProperties": false, "required": ["label", "facts"], "properties": {"label": {"type": "string"}, "facts": {"type": "object"}}}}, "usageProfiles": {"type": "array", "items": {"$ref": "#/$defs/usageProfile"}}, "auth": {}
|
||||
},
|
||||
"allOf": [{"if": {"required": ["requiredCapabilities"], "properties": {"requiredCapabilities": {"contains": {"const": "submit-sse-delta@1"}}}}, "then": {"required": ["submitResponseTypes"], "properties": {"submitResponseTypes": {"contains": {"const": "sse"}}}}}],
|
||||
|
||||
@@ -15,6 +15,7 @@ const (
|
||||
CapabilityJSONClone = "json-clone@1"
|
||||
CapabilitySubmitSSEDelta = "submit-sse-delta@1"
|
||||
CapabilityJSONOrder = "json-order@1"
|
||||
CapabilityDurationAuto = "duration-auto@1"
|
||||
MaxJSONToolBytes = 1 << 20
|
||||
maxJSONToolDepth = 32
|
||||
maxJSONToolNodes = 32768
|
||||
@@ -23,7 +24,7 @@ const (
|
||||
|
||||
// HasCapability describes host APIs independently of a plugin's mutable globals.
|
||||
func HasCapability(name string) bool {
|
||||
return name == CapabilityJSONClone || name == CapabilitySubmitSSEDelta || name == CapabilityJSONOrder
|
||||
return name == CapabilityJSONClone || name == CapabilitySubmitSSEDelta || name == CapabilityJSONOrder || name == CapabilityDurationAuto
|
||||
}
|
||||
|
||||
// JSONState owns a request-local JSON result. Appended strings stay in Go and
|
||||
|
||||
@@ -132,6 +132,14 @@ func (m Meta) PreservesJSONOrder() bool {
|
||||
return slices.Contains(m.RequiredCapabilities, CapabilityJSONOrder)
|
||||
}
|
||||
|
||||
// AcceptsAutoDuration reports whether the plugin declared duration-auto@1:
|
||||
// request validation lets a duration of -1, the vendor's "the model picks the
|
||||
// length", through to its hooks. The plugin reserves the longest duration the
|
||||
// model can produce and settles the task on the usage the vendor reports.
|
||||
func (m Meta) AcceptsAutoDuration() bool {
|
||||
return slices.Contains(m.RequiredCapabilities, CapabilityDurationAuto)
|
||||
}
|
||||
|
||||
// JSONTextMember names the member of a hook result (a decoded requestBody, a
|
||||
// request descriptor's body) that the host also takes as JSON text, for a
|
||||
// plugin that preserves JSON order; it is empty for other plugins.
|
||||
|
||||
@@ -38,7 +38,7 @@ func TestAlibabaWan3(t *testing.T) {
|
||||
return roundTrip(t, value), nil
|
||||
}
|
||||
|
||||
t.Run("duration -1 becomes the auto_duration marker so the host accepts the body", func(t *testing.T) {
|
||||
t.Run("duration -1 becomes the auto_duration marker in every decoder", func(t *testing.T) {
|
||||
resolved, callErr := decodeResponses("wan3.0-video", map[string]any{"model": "wan3.0-video", "input": "a cat", "duration": -1})
|
||||
require.NoError(t, callErr)
|
||||
requestBody := resolved["requestBody"].(map[string]any)
|
||||
@@ -83,6 +83,16 @@ func TestAlibabaWan3(t *testing.T) {
|
||||
require.ErrorContains(t, callErr, "only supported by wan3.0")
|
||||
})
|
||||
|
||||
// /v1/video/generations has no plugin decoder, so -1 reaches the host's
|
||||
// request validation as a vendor parameter; duration-auto@1 lets it through.
|
||||
t.Run("metadata parameters keep -1 on /v1/video/generations", func(t *testing.T) {
|
||||
body, facts, _ := submitAlibabaRequest(t, plugin, "wan3.0-video", map[string]any{
|
||||
"model": "wan3.0-video", "prompt": "a cat", "metadata": map[string]any{"parameters": map[string]any{"duration": -1}},
|
||||
})
|
||||
assert.Equal(t, float64(-1), body["parameters"].(map[string]any)["duration"])
|
||||
assert.Equal(t, float64(30), facts["seconds"])
|
||||
})
|
||||
|
||||
t.Run("channel-mapped alias resolves defaults from the upstream model", func(t *testing.T) {
|
||||
direct, callErr := plugin.Engine.Call(t.Context(), "buildSubmitRequest", submitCtx("wan3.0-video", "wan3.0-video", map[string]any{"model": "wan3.0-video", "prompt": "a cat"}))
|
||||
require.NoError(t, callErr)
|
||||
|
||||
@@ -393,6 +393,67 @@ func TestDoubaoSeedanceUsageFacts(t *testing.T) {
|
||||
})
|
||||
}
|
||||
|
||||
// Ark reads duration -1 as "the model picks the length" (Seedance 1.5 pro
|
||||
// and 2.x; 2.5 also defaults to it). The reservation covers the longest
|
||||
// output: 720p is 1280 × 720 × 24 / 1024 = 21,600 tokens per second.
|
||||
t.Run("duration -1 reaches Ark and reserves the longest duration", func(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
model string
|
||||
request any
|
||||
wantDuration any
|
||||
wantTokens float64
|
||||
}{
|
||||
{"2.5 metadata on /v1/video/generations", v25, relaycommon.TaskSubmitReq{
|
||||
Model: v25, Prompt: "a cat", Metadata: map[string]any{"duration": -1, "resolution": "720p"},
|
||||
}, float64(-1), 30 * 21600},
|
||||
{"2.5 without a duration keeps the Ark default", v25, map[string]any{
|
||||
"model": v25, "prompt": "a cat", "metadata": map[string]any{"resolution": "720p"},
|
||||
}, nil, 30 * 21600},
|
||||
{"2.0 seconds from Responses or /v1/videos", v20, map[string]any{
|
||||
"model": v20, "prompt": "a cat", "seconds": -1, "metadata": map[string]any{"resolution": "720p"},
|
||||
}, float64(-1), 15 * 21600},
|
||||
{"1.5 pro", pro15, map[string]any{
|
||||
"model": pro15, "prompt": "a cat", "metadata": map[string]any{"duration": -1, "resolution": "720p"},
|
||||
}, float64(-1), 12 * 21600},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
info := &relaycommon.RelayInfo{
|
||||
ChannelMeta: &relaycommon.ChannelMeta{ChannelBaseUrl: doubaoBaseURL, UpstreamModelName: tc.model},
|
||||
OriginModelName: tc.model,
|
||||
TaskRelayInfo: &relaycommon.TaskRelayInfo{PublicTaskID: "task_public", Action: "text_to_video"},
|
||||
}
|
||||
adaptor := taskplugin.New(plugin)
|
||||
adaptor.Init(info)
|
||||
c, _ := gin.CreateTestContext(httptest.NewRecorder())
|
||||
c.Request = httptest.NewRequest(http.MethodPost, "/v1/video/generations", nil)
|
||||
c.Set("task_request", tc.request)
|
||||
require.Nil(t, adaptor.ValidateRequestAndSetAction(c, info))
|
||||
reader, err := adaptor.BuildRequestBody(c, info)
|
||||
require.NoError(t, err)
|
||||
encoded, err := io.ReadAll(reader)
|
||||
require.NoError(t, err)
|
||||
var body map[string]any
|
||||
require.NoError(t, common.Unmarshal(encoded, &body))
|
||||
assert.Equal(t, tc.wantDuration, body["duration"])
|
||||
facts, err := adaptor.ExtractUsageFactsValidated(c, info)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, tc.wantTokens, facts["tokens"])
|
||||
})
|
||||
}
|
||||
|
||||
decode := func(body map[string]any) (any, error) {
|
||||
return plugin.Engine.CallPath(t.Context(), "protocols", []string{"openai_video", "decodeRequest"}, map[string]any{
|
||||
"model": v20, "body": map[string]any{"kind": "json", "value": body},
|
||||
})
|
||||
}
|
||||
value, err := decode(map[string]any{"model": v20, "prompt": "a cat", "duration": -1})
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, float64(-1), alibabaObject(t, value)["requestBody"].(map[string]any)["seconds"])
|
||||
_, err = decode(map[string]any{"model": v20, "prompt": "a cat", "seconds": -2})
|
||||
require.ErrorContains(t, err, "seconds must be -1 or between 1 and 3600")
|
||||
})
|
||||
|
||||
t.Run("completion overlays only resolutions the model offers", func(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
model string
|
||||
|
||||
@@ -222,7 +222,7 @@ export const meta = {
|
||||
en: "Alibaba Cloud Bailian image and video generation (Wan, Qwen-Image, Z-Image)",
|
||||
zh: "阿里云百炼图片与视频生成(万相、千问图像、Z-Image)",
|
||||
},
|
||||
version: "1.4.1",
|
||||
version: "1.4.2",
|
||||
author: { name: "QuantumNous" },
|
||||
channelTypes: [17],
|
||||
// Literal metadata also supports the dashboard's static script preview.
|
||||
@@ -277,7 +277,7 @@ export const meta = {
|
||||
fetchMode: "per_task",
|
||||
upstreams: ["vendor", "new_api"],
|
||||
submitResponseTypes: ["json", "sse"],
|
||||
requiredCapabilities: ["json-clone@1", "submit-sse-delta@1"],
|
||||
requiredCapabilities: ["json-clone@1", "submit-sse-delta@1", "duration-auto@1"],
|
||||
usageSchema: { ...WAN_IMAGE_USAGE_SCHEMA, ...wanVideoUsageSchema(WAN_VIDEO_RESOLUTIONS, false) },
|
||||
usageProfiles: [
|
||||
{
|
||||
@@ -703,8 +703,9 @@ function objectValue(value, name) {
|
||||
return value;
|
||||
}
|
||||
|
||||
// The host rejects negative canonical duration/seconds facts before any hook
|
||||
// runs, so wan3.0's "-1 = smart duration" sentinel travels as a boolean marker.
|
||||
// Decoders fold wan3.0's "-1 = smart duration" sentinel, wherever the client
|
||||
// put it, into one boolean marker; convert sends -1 upstream and extractUsage
|
||||
// reserves 30 seconds, the longest wan3.0 output, for it.
|
||||
function normalizeRequest(value) {
|
||||
const req = Object.assign({}, value);
|
||||
const metadata = objectValue(req.metadata, "metadata");
|
||||
|
||||
@@ -1,22 +1,24 @@
|
||||
// Seedance capabilities from the Ark model list, not prices: the output
|
||||
// resolutions each model offers, whether it accepts reference video input
|
||||
// (Seedance 2.x prices video-to-video tokens separately) and whether audio
|
||||
// output is priced separately (Seedance 1.5 pro; generate_audio defaults to
|
||||
// true). Seedance 1.0 lite is no longer listed and keeps the 1.0 pro tiers.
|
||||
// resolutions each model offers, the longest output duration in seconds
|
||||
// (duration -1 lets Seedance 1.5 pro and 2.x pick a length up to it), whether
|
||||
// it accepts reference video input (Seedance 2.x prices video-to-video tokens
|
||||
// separately) and whether audio output is priced separately (Seedance 1.5 pro;
|
||||
// generate_audio defaults to true). Seedance 1.0 lite is no longer listed and
|
||||
// keeps the 1.0 pro tiers.
|
||||
const VIDEO_MODELS = {
|
||||
"doubao-seedance-1-0-pro-250528": { resolutions: ["480p", "720p", "1080p"] },
|
||||
"doubao-seedance-1-0-lite-t2v": { resolutions: ["480p", "720p", "1080p"] },
|
||||
"doubao-seedance-1-0-lite-i2v": { resolutions: ["480p", "720p", "1080p"] },
|
||||
"doubao-seedance-1-5-pro-251215": { resolutions: ["480p", "720p", "1080p"], audio: true },
|
||||
"doubao-seedance-2-0-260128": { resolutions: ["480p", "720p", "1080p", "4k"], videoInput: true },
|
||||
"doubao-seedance-2-0-fast-260128": { resolutions: ["480p", "720p"], videoInput: true },
|
||||
"doubao-seedance-2-0-mini-260615": { resolutions: ["480p", "720p"], videoInput: true },
|
||||
"doubao-seedance-2-5-260628": { resolutions: ["480p", "720p", "1080p"], videoInput: true },
|
||||
"doubao-seedance-1-0-pro-250528": { resolutions: ["480p", "720p", "1080p"], maxDuration: 12 },
|
||||
"doubao-seedance-1-0-lite-t2v": { resolutions: ["480p", "720p", "1080p"], maxDuration: 12 },
|
||||
"doubao-seedance-1-0-lite-i2v": { resolutions: ["480p", "720p", "1080p"], maxDuration: 12 },
|
||||
"doubao-seedance-1-5-pro-251215": { resolutions: ["480p", "720p", "1080p"], maxDuration: 12, audio: true },
|
||||
"doubao-seedance-2-0-260128": { resolutions: ["480p", "720p", "1080p", "4k"], maxDuration: 15, videoInput: true },
|
||||
"doubao-seedance-2-0-fast-260128": { resolutions: ["480p", "720p"], maxDuration: 15, videoInput: true },
|
||||
"doubao-seedance-2-0-mini-260615": { resolutions: ["480p", "720p"], maxDuration: 15, videoInput: true },
|
||||
"doubao-seedance-2-5-260628": { resolutions: ["480p", "720p", "1080p"], maxDuration: 30, videoInput: true },
|
||||
};
|
||||
// Every Ark resolution tier; an endpoint ID reached through channel mapping
|
||||
// without a declared profile keeps them all.
|
||||
// without a declared profile keeps them all and the longest Seedance duration.
|
||||
const SEEDANCE_RESOLUTIONS = ["480p", "720p", "1080p", "4k"];
|
||||
const DEFAULT_VIDEO_PROFILE = { resolutions: SEEDANCE_RESOLUTIONS, videoInput: true };
|
||||
const DEFAULT_VIDEO_PROFILE = { resolutions: SEEDANCE_RESOLUTIONS, maxDuration: 30, videoInput: true };
|
||||
|
||||
// Protocol capabilities from the Ark image generation API reference, not prices.
|
||||
// presets: accepted `size` resolution tiers; minPixels/maxPixels bound `WxH` sizes.
|
||||
@@ -301,12 +303,13 @@ export const meta = {
|
||||
en: "Volcengine Doubao Seedance video generation and Seedream image generation",
|
||||
zh: "火山引擎豆包 Seedance 视频生成与 Seedream 图片生成",
|
||||
},
|
||||
version: "1.2.1",
|
||||
version: "1.3.0",
|
||||
author: { name: "QuantumNous" },
|
||||
channelTypes: [54, 45], // VolcEngine-type channels serve Ark video models with the same wire format
|
||||
models: Object.keys(VIDEO_MODELS).concat(Object.keys(IMAGE_MODELS)),
|
||||
fetchMode: "per_task",
|
||||
upstreams: ["vendor", "new_api"],
|
||||
requiredCapabilities: ["duration-auto@1"],
|
||||
usageSchema: seedanceUsageSchema(DEFAULT_VIDEO_PROFILE),
|
||||
usageExamples: seedanceUsageExamples(DEFAULT_VIDEO_PROFILE),
|
||||
usageProfiles: capabilityUsageProfiles(VIDEO_MODELS, seedanceUsageSchema, seedanceUsageExamples).concat(
|
||||
@@ -811,8 +814,9 @@ export function buildSubmitRequest(ctx) {
|
||||
const hasReference = body.content.length > 0;
|
||||
if (trimmed(req.prompt) || !hasReference) body.content.push({ type: "text", text: req.prompt || "" });
|
||||
if (Array.isArray(body.content)) body.content = rewriteDraftTaskContent(body.content, ctx.originTasks);
|
||||
// -1 lets the model pick the length; extractUsage reserves the longest one.
|
||||
const seconds = Number.parseInt(req.seconds || "", 10);
|
||||
if (seconds > 0) body.duration = seconds;
|
||||
if (seconds > 0 || seconds === -1) body.duration = seconds;
|
||||
body.model = ctx.upstreamModel || body.model;
|
||||
return {
|
||||
url: apiRoot(ctx) + "/api/v3/contents/generations/tasks",
|
||||
@@ -856,14 +860,16 @@ export function extractUsage(ctx) {
|
||||
const ratio = videoInputRatio(ctx.upstreamModel || ctx.model, metadata.resolution, metadata.content);
|
||||
return ratio === 1 ? null : { video_input_ratio: ratio };
|
||||
}
|
||||
const profile = videoProfile(ctx);
|
||||
// A missing duration or -1 (the model picks the length; Seedance 2.5
|
||||
// defaults to it) reserves the longest duration the model can produce.
|
||||
let seconds = Number(req.seconds || req.duration || metadata.duration || 0);
|
||||
if (!Number.isFinite(seconds) || seconds <= 0) {
|
||||
const frames = Number(metadata.frames);
|
||||
seconds = Number.isFinite(frames) && frames > 0 ? Math.floor(frames / 24) : 15;
|
||||
seconds = Number.isFinite(frames) && frames > 0 ? Math.floor(frames / 24) : profile.maxDuration;
|
||||
}
|
||||
if (seconds <= 0) seconds = 5;
|
||||
seconds = Math.min(seconds, 3600);
|
||||
const profile = videoProfile(ctx);
|
||||
const resolution = videoResolution(ctx);
|
||||
const facts = { tokens: estimateTokens(seconds, resolution), resolution: resolution };
|
||||
// Every model reports video_input, including those whose profile omits it:
|
||||
@@ -1154,13 +1160,16 @@ protocols.openai_video = {
|
||||
if (!ctx.body.value || Array.isArray(ctx.body.value)) throw new Error("JSON object required");
|
||||
const req = ctx.body.value;
|
||||
const seconds = req.seconds === undefined ? req.duration : req.seconds;
|
||||
if (seconds !== undefined && (!Number.isFinite(Number(seconds)) || Number(seconds) <= 0 || Number(seconds) > 3600))
|
||||
throw new Error("seconds must be between 1 and 3600");
|
||||
if (seconds !== undefined && Number(seconds) !== -1 && (!Number.isFinite(Number(seconds)) || Number(seconds) <= 0 || Number(seconds) > 3600))
|
||||
throw new Error("seconds must be -1 or between 1 and 3600");
|
||||
const requestBody = Object.assign({}, req, { model: ctx.model });
|
||||
// buildSubmitRequest reads seconds; duration is accepted as its alias.
|
||||
if (seconds !== undefined) requestBody.seconds = seconds;
|
||||
return {
|
||||
kind: "submit",
|
||||
model: ctx.model,
|
||||
action: req.input_reference || req.image ? "image_to_video" : "text_to_video",
|
||||
requestBody: Object.assign({}, req, { model: ctx.model }),
|
||||
requestBody: requestBody,
|
||||
};
|
||||
}
|
||||
const first = function (name) {
|
||||
@@ -1187,8 +1196,8 @@ protocols.openai_video = {
|
||||
if (req.seconds !== undefined) req.seconds = Number(req.seconds);
|
||||
else if (req.duration !== undefined) req.seconds = Number(req.duration);
|
||||
const seconds = req.seconds === undefined ? req.duration : req.seconds;
|
||||
if (seconds !== undefined && (!Number.isFinite(Number(seconds)) || Number(seconds) <= 0 || Number(seconds) > 3600))
|
||||
throw new Error("seconds must be between 1 and 3600");
|
||||
if (seconds !== undefined && Number(seconds) !== -1 && (!Number.isFinite(Number(seconds)) || Number(seconds) <= 0 || Number(seconds) > 3600))
|
||||
throw new Error("seconds must be -1 or between 1 and 3600");
|
||||
return {
|
||||
kind: "submit",
|
||||
model: ctx.model,
|
||||
|
||||
@@ -1521,11 +1521,21 @@ func (a *TaskAdaptor) validateResolvedUsageValue(value any, usageSchema map[stri
|
||||
switch typed := value.(type) {
|
||||
case map[string]any:
|
||||
for key, item := range typed {
|
||||
limit, canonical := canonicalUsageLimit(key)
|
||||
// A duration of -1 asks the vendor to pick the length. Plugins that
|
||||
// declare duration-auto@1 reserve the longest duration for it and
|
||||
// settle on the usage the vendor reports; usage facts stay
|
||||
// non-negative.
|
||||
if canonical && limit == relaycommon.MaxTaskDurationSeconds && a.plugin.Meta.AcceptsAutoDuration() {
|
||||
if number, _ := usageNumber(item, true); number == -1 {
|
||||
continue
|
||||
}
|
||||
}
|
||||
if schema, declared := usageSchema[key]; declared {
|
||||
if _, err := validateUsageValue(item, schema, true); err != nil {
|
||||
return err
|
||||
}
|
||||
} else if limit, canonical := canonicalUsageLimit(key); canonical {
|
||||
} else if canonical {
|
||||
if err := validateUsageLimit(item, limit, true); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -941,6 +941,53 @@ export function extractUsageOnComplete(task, result, body) { return (body || {})
|
||||
assert.Nil(t, ratios)
|
||||
})
|
||||
|
||||
t.Run("duration -1 passes only for plugins that declare duration-auto@1", func(t *testing.T) {
|
||||
declaring, err := pluginruntime.NewRegistry().Register(strings.Replace(source, `fetchMode: "per_task",`, `fetchMode: "per_task", requiredCapabilities: ["duration-auto@1"],`, 1), pluginruntime.Options{})
|
||||
require.NoError(t, err)
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
declaring bool
|
||||
body map[string]any
|
||||
accepted bool
|
||||
}{
|
||||
{"undeclared plugins keep rejecting it", false, map[string]any{"metadata": map[string]any{"duration": -1}}, false},
|
||||
{"vendor parameters and declared usage fields", true, map[string]any{
|
||||
"duration": -1,
|
||||
"metadata": map[string]any{"seconds": "-1", "parameters": map[string]any{"durationSeconds": -1}},
|
||||
}, true},
|
||||
{"other negative durations", true, map[string]any{"metadata": map[string]any{"duration": -2}}, false},
|
||||
{"durations above the host limit", true, map[string]any{"metadata": map[string]any{"duration": relaycommon.MaxTaskDurationSeconds + 1}}, false},
|
||||
{"negative counts", true, map[string]any{"metadata": map[string]any{"count": -1}}, false},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
adaptor, context, info := newRequest(t, tc.body)
|
||||
if tc.declaring {
|
||||
adaptor = New(declaring)
|
||||
adaptor.Init(info)
|
||||
}
|
||||
taskErr := adaptor.ValidateRequestAndSetAction(context, info)
|
||||
if tc.accepted {
|
||||
assert.Nil(t, taskErr)
|
||||
return
|
||||
}
|
||||
require.NotNil(t, taskErr)
|
||||
assert.Equal(t, "plugin_usage_invalid", taskErr.Code)
|
||||
})
|
||||
}
|
||||
|
||||
adaptor, context, info := newRequest(t, map[string]any{
|
||||
"metadata": map[string]any{"duration": -1},
|
||||
"hookUsageEntries": []any{map[string]any{"name": "duration", "value": -1}},
|
||||
})
|
||||
adaptor = New(declaring)
|
||||
adaptor.Init(info)
|
||||
require.Nil(t, adaptor.ValidateRequestAndSetAction(context, info))
|
||||
_, err = adaptor.ExtractUsageFactsValidated(context, info)
|
||||
assert.Error(t, err, "usage facts stay non-negative")
|
||||
_, err = adaptor.EstimateBillingValidated(context, info)
|
||||
assert.Error(t, err, "billing ratios stay non-negative")
|
||||
})
|
||||
|
||||
t.Run("declared token facts use int32 saturation instead of duration cap", func(t *testing.T) {
|
||||
adaptor, context, info := newRequest(t, map[string]any{
|
||||
"hookUsageEntries": []any{
|
||||
|
||||
Reference in New Issue
Block a user