mirror of
https://gh-proxy.org/https://github.com/klinsmaya/open-connector.git
synced 2026-11-08 01:16:52 +08:00
l1shen
cdd4b5970f
Self-hosted Open Flow needs OpenConnector to execute provider Triggers and own their remote subscriptions. This adds a native Trigger runtime for 21 Triggers across 18 providers, including polling, webhook delivery, configuration options, and Feishu resource subscriptions. ## Changes - Register provider Trigger definitions and lazy executors, expose permission guidance and the five supported operations (`options`, `read`, `reconcile`, `receive`, `resource`), and export the catalog consumed by Open Flow. - Add independent deployment, runtime, and token Trigger grants. Existing persistent tokens default to an empty Trigger grant; stateful operations require a persistent token. Trigger-only grants authorize registered provider calls through the existing authenticated, SSRF-guarded transport. - Resolve `x-oo-connector-app-id` as a stable connection ID for Triggers, Actions, and public proxy. Conflicting selectors fail, and unknown IDs cannot fall back to another account. Runtime connection metadata exposes verified provider account identity. - Persist subscription ownership, callback nonces, provider state, and cleanup status using the existing secret codec and SQL leases across SQLite, PostgreSQL, and D1. Enforce immutable active bindings, validate callbacks, and share Feishu resource references by provider account across connection aliases and tokens. - Protect connections while remote resources still need cleanup, allow verified same-account reauthorization to recover cleanup credentials and rebuild released resource bindings with the current connection revision, and provide administrator subscription inspection, cancellation, and explicit abandonment. - Run bounded cleanup after grant or token revocation in Node and Workers. Open Flow owns public ingress, scheduling, business checkpoints, deduplication, and active subscription reconciliation. - Bound Poll event pages to 100 events, preserve provider continuation checkpoints, and align native Google Drive requests with the registered proxy. Gmail resumes within a history page before advancing its history cursor; Google Sheets reads from the next unread row with an inclusive window that respects the event budget. - Document the API, permissions, lifecycle, and headless packaging contracts. Preserve Trigger grants through Web Console token edits. - Fix the PGlite socket fixture's premature `ReadyForQuery` responses after extended-query errors, with regression coverage for connection recovery after both Parse and Execute failures. ## Deployment notes Apply migrations `0016_trigger_policy` and `0017_trigger_subscriptions` before starting a PostgreSQL runtime. Grant `allowedTriggers` explicitly to persistent tokens that manage subscriptions. Native local connections are supported; SaaS and Marketplace Trigger requests return `trigger_source_not_supported`. ## Validation - `npm run fix-check`: passed lint, formatting, and type checks. - `npm test`: 3435 passed, 20 skipped. - PostgreSQL suite: 10 consecutive runs passed, 38 tests per run. Both new protocol regression cases fail against the uncorrected fixture. - Resource HTTP regressions cover cross-alias/token release and abandonment, account isolation, and same-account credential rotation. - Native provider regressions use the registered proxies and drain 250-event backlogs under default, small, and large page settings. All 24 provider contract tests also pass against the compiled modules installed from the actual tarball. New cases cover empty and populated Sheets seeds, Gmail history records containing more messages than the event budget, filtering, deleted messages, transient failures, and JSON checkpoint persistence. All 11 new cases fail against the preceding implementation. - Generated catalog: all 21 Trigger contracts match the Open Flow consumer catalog. - Runtime package build and isolated tarball consumption: passed, including native GitHub subscription creation and cleanup against a fake upstream. - Compiled host smoke checks: passed with all providers, GitHub only, and an empty provider selection. - Web build and Workers deployment dry run: passed. - Cross-repository HTTP integration: GitHub and Feishu publish, callback-to-run delivery, provider account identity, and cleanup after token revocation passed against fake upstreams. Real third-party account acceptance remains unverified. Workers validation covers build and D1 behavior; it does not include a Cloudflare deployment.
31 lines
1.5 KiB
TypeScript
31 lines
1.5 KiB
TypeScript
import type { ExecutorModules } from "../src/providers/provider-loader.ts";
|
|
|
|
import { writeFile } from "node:fs/promises";
|
|
import { isDeepStrictEqual } from "node:util";
|
|
import { ProviderLoader } from "../src/providers/provider-loader.ts";
|
|
import { loadProviderSources } from "./provider-source.ts";
|
|
|
|
const target = process.argv[2];
|
|
if (!target) throw new Error("Pass the Open Flow catalog output path.");
|
|
const registryPath = "../src/providers/registry.generated.ts";
|
|
const { executorModules } = (await import(registryPath)) as { executorModules: ExecutorModules };
|
|
const loader = new ProviderLoader(executorModules);
|
|
const entries = [];
|
|
for (const { definition: provider } of await loadProviderSources()) {
|
|
if (!provider.triggers?.length) continue;
|
|
for (const trigger of await loader.loadTriggerDefinitions(provider.service)) {
|
|
const snapshot = provider.triggers.find((item) => item.key === trigger.snapshot.key);
|
|
if (!snapshot || !isDeepStrictEqual(snapshot, trigger.snapshot))
|
|
throw new Error(`Trigger metadata drift: ${trigger.snapshot.key}`);
|
|
entries.push({
|
|
snapshot,
|
|
options: trigger.configOptions != null,
|
|
intervalMs: "listener" in trigger ? trigger.listener?.intervalMs : undefined,
|
|
eventSource: "eventSource" in trigger ? trigger.eventSource : undefined,
|
|
});
|
|
}
|
|
}
|
|
entries.sort((a, b) => a.snapshot.key.localeCompare(b.snapshot.key));
|
|
await writeFile(target, JSON.stringify(entries, null, 2) + "\n");
|
|
console.log(`Exported ${entries.length} Trigger definitions to ${target}.`);
|