l1shen
cdd4b5970f
Self-hosted Open Flow needs OpenConnector to execute provider Triggers and own their remote subscriptions. This adds a native Trigger runtime for 21 Triggers across 18 providers, including polling, webhook delivery, configuration options, and Feishu resource subscriptions. ## Changes - Register provider Trigger definitions and lazy executors, expose permission guidance and the five supported operations (`options`, `read`, `reconcile`, `receive`, `resource`), and export the catalog consumed by Open Flow. - Add independent deployment, runtime, and token Trigger grants. Existing persistent tokens default to an empty Trigger grant; stateful operations require a persistent token. Trigger-only grants authorize registered provider calls through the existing authenticated, SSRF-guarded transport. - Resolve `x-oo-connector-app-id` as a stable connection ID for Triggers, Actions, and public proxy. Conflicting selectors fail, and unknown IDs cannot fall back to another account. Runtime connection metadata exposes verified provider account identity. - Persist subscription ownership, callback nonces, provider state, and cleanup status using the existing secret codec and SQL leases across SQLite, PostgreSQL, and D1. Enforce immutable active bindings, validate callbacks, and share Feishu resource references by provider account across connection aliases and tokens. - Protect connections while remote resources still need cleanup, allow verified same-account reauthorization to recover cleanup credentials and rebuild released resource bindings with the current connection revision, and provide administrator subscription inspection, cancellation, and explicit abandonment. - Run bounded cleanup after grant or token revocation in Node and Workers. Open Flow owns public ingress, scheduling, business checkpoints, deduplication, and active subscription reconciliation. - Bound Poll event pages to 100 events, preserve provider continuation checkpoints, and align native Google Drive requests with the registered proxy. Gmail resumes within a history page before advancing its history cursor; Google Sheets reads from the next unread row with an inclusive window that respects the event budget. - Document the API, permissions, lifecycle, and headless packaging contracts. Preserve Trigger grants through Web Console token edits. - Fix the PGlite socket fixture's premature `ReadyForQuery` responses after extended-query errors, with regression coverage for connection recovery after both Parse and Execute failures. ## Deployment notes Apply migrations `0016_trigger_policy` and `0017_trigger_subscriptions` before starting a PostgreSQL runtime. Grant `allowedTriggers` explicitly to persistent tokens that manage subscriptions. Native local connections are supported; SaaS and Marketplace Trigger requests return `trigger_source_not_supported`. ## Validation - `npm run fix-check`: passed lint, formatting, and type checks. - `npm test`: 3435 passed, 20 skipped. - PostgreSQL suite: 10 consecutive runs passed, 38 tests per run. Both new protocol regression cases fail against the uncorrected fixture. - Resource HTTP regressions cover cross-alias/token release and abandonment, account isolation, and same-account credential rotation. - Native provider regressions use the registered proxies and drain 250-event backlogs under default, small, and large page settings. All 24 provider contract tests also pass against the compiled modules installed from the actual tarball. New cases cover empty and populated Sheets seeds, Gmail history records containing more messages than the event budget, filtering, deleted messages, transient failures, and JSON checkpoint persistence. All 11 new cases fail against the preceding implementation. - Generated catalog: all 21 Trigger contracts match the Open Flow consumer catalog. - Runtime package build and isolated tarball consumption: passed, including native GitHub subscription creation and cleanup against a fake upstream. - Compiled host smoke checks: passed with all providers, GitHub only, and an empty provider selection. - Web build and Workers deployment dry run: passed. - Cross-repository HTTP integration: GitHub and Feishu publish, callback-to-run delivery, provider account identity, and cleanup after token revocation passed against fake upstreams. Real third-party account acceptance remains unverified. Workers validation covers build and D1 behavior; it does not include a Cloudflare deployment.
2.6 KiB
Catalog Format
Provider definitions in src/providers/<service>/definition.ts are the source of truth.
npm run generate:catalog writes catalog/apps/<service>.json and the startup index
catalog/apps-index.json together; both are generated local runtime data used by the server at
startup, and neither must be hand-edited. Generated registry and catalog files are ignored by git.
npm install, npm run dev, and npm start create them when they are missing or stale.
Provider executors live in src/providers/<service>/executors.ts and are loaded only when an action is executed.
Do not hand-edit generated catalog files as source. Update provider definitions and run:
npm run generate:catalog
At runtime, catalog responses add execution status that is not stored in generated catalog JSON:
locallyExecutable: the open-source runtime has a local executor for the action.catalogOnly: schemas and metadata are available, but no local executor is wired yet.needsCredential: the provider needs a configured local connection before execution.noAuthRunnable: the action belongs to a provider that can run without stored credentials.
Action definitions declare a required operationType of read, write, or destructive. This is
the provider-side state effect of the action's most powerful valid input, not a risk level or an
authorization rule. Definitions also declare provider-native requiredScopes and
providerPermissions. The runtime exposes these fields through HTTP and MCP discovery together
with the current connection profile, so agents can see both the capability they are about to use
and the account it will run as.
For the full contribution workflow, see .codex/skills/add-provider/SKILL.md.
Trigger metadata
Providers with native Trigger implementations publish triggers snapshots and triggerPermissions in their generated catalog entries. Snapshots keep configuration ports, outputs, identity and definition version; permission guidance uses provider-native scopes. Execution modules export registered Trigger definitions and load through the existing lazy executor registry.
To update an Open Flow checkout without a private repository, install this repository's dependencies and run:
node scripts/export-flow-trigger-catalog.ts /path/to/open-flow/packages/open-flow/src/trigger/providers/catalog.generated.json
The export includes configuration-option support, listener intervals and event-source ownership, and checks that runtime snapshots match the public provider definitions. Open Flow's checked-in snapshot supports independent builds; regeneration requires only a public OpenConnector checkout.