Self-hosted Open Flow needs OpenConnector to execute provider Triggers and own their remote subscriptions. This adds a native Trigger runtime for 21 Triggers across 18 providers, including polling, webhook delivery, configuration options, and Feishu resource subscriptions. ## Changes - Register provider Trigger definitions and lazy executors, expose permission guidance and the five supported operations (`options`, `read`, `reconcile`, `receive`, `resource`), and export the catalog consumed by Open Flow. - Add independent deployment, runtime, and token Trigger grants. Existing persistent tokens default to an empty Trigger grant; stateful operations require a persistent token. Trigger-only grants authorize registered provider calls through the existing authenticated, SSRF-guarded transport. - Resolve `x-oo-connector-app-id` as a stable connection ID for Triggers, Actions, and public proxy. Conflicting selectors fail, and unknown IDs cannot fall back to another account. Runtime connection metadata exposes verified provider account identity. - Persist subscription ownership, callback nonces, provider state, and cleanup status using the existing secret codec and SQL leases across SQLite, PostgreSQL, and D1. Enforce immutable active bindings, validate callbacks, and share Feishu resource references by provider account across connection aliases and tokens. - Protect connections while remote resources still need cleanup, allow verified same-account reauthorization to recover cleanup credentials and rebuild released resource bindings with the current connection revision, and provide administrator subscription inspection, cancellation, and explicit abandonment. - Run bounded cleanup after grant or token revocation in Node and Workers. Open Flow owns public ingress, scheduling, business checkpoints, deduplication, and active subscription reconciliation. - Bound Poll event pages to 100 events, preserve provider continuation checkpoints, and align native Google Drive requests with the registered proxy. Gmail resumes within a history page before advancing its history cursor; Google Sheets reads from the next unread row with an inclusive window that respects the event budget. - Document the API, permissions, lifecycle, and headless packaging contracts. Preserve Trigger grants through Web Console token edits. - Fix the PGlite socket fixture's premature `ReadyForQuery` responses after extended-query errors, with regression coverage for connection recovery after both Parse and Execute failures. ## Deployment notes Apply migrations `0016_trigger_policy` and `0017_trigger_subscriptions` before starting a PostgreSQL runtime. Grant `allowedTriggers` explicitly to persistent tokens that manage subscriptions. Native local connections are supported; SaaS and Marketplace Trigger requests return `trigger_source_not_supported`. ## Validation - `npm run fix-check`: passed lint, formatting, and type checks. - `npm test`: 3435 passed, 20 skipped. - PostgreSQL suite: 10 consecutive runs passed, 38 tests per run. Both new protocol regression cases fail against the uncorrected fixture. - Resource HTTP regressions cover cross-alias/token release and abandonment, account isolation, and same-account credential rotation. - Native provider regressions use the registered proxies and drain 250-event backlogs under default, small, and large page settings. All 24 provider contract tests also pass against the compiled modules installed from the actual tarball. New cases cover empty and populated Sheets seeds, Gmail history records containing more messages than the event budget, filtering, deleted messages, transient failures, and JSON checkpoint persistence. All 11 new cases fail against the preceding implementation. - Generated catalog: all 21 Trigger contracts match the Open Flow consumer catalog. - Runtime package build and isolated tarball consumption: passed, including native GitHub subscription creation and cleanup against a fake upstream. - Compiled host smoke checks: passed with all providers, GitHub only, and an empty provider selection. - Web build and Workers deployment dry run: passed. - Cross-repository HTTP integration: GitHub and Feishu publish, callback-to-run delivery, provider account identity, and cleanup after token revocation passed against fake upstreams. Real third-party account acceptance remains unverified. Workers validation covers build and D1 behavior; it does not include a Cloudflare deployment.
7.9 KiB
@oomol-lab/open-connector
Embeddable Open Connector runtime for Node.js 22.18+ and Bun 1.4+.
This package is the headless runtime library. It owns providers, authorization, credentials, and storage. Your application owns the HTTP server, configuration, and process lifecycle.
It ships compiled JavaScript, TypeScript types, the provider catalog, and SQL migrations. It does not include the Web Console or install scripts. For a standalone server with a dashboard, use the Docker image or a single-file binary.
Install
npm install @oomol-lab/open-connector @hono/node-server
Quick start
import { serve } from "@hono/node-server";
import { createConnectorRuntime } from "@oomol-lab/open-connector";
const connector = await createConnectorRuntime({
dataDir: "./data/connector",
publicOrigin: "https://app.example.com/connector",
encryptionKey: process.env.CONNECTOR_ENCRYPTION_KEY,
adminToken: process.env.CONNECTOR_ADMIN_TOKEN,
runtimeToken: process.env.CONNECTOR_RUNTIME_TOKEN,
});
const server = serve({
fetch: (request) => connector.fetch(request),
port: 3000,
});
const shutdown = async () => {
server.close();
await connector.close();
};
process.on("SIGINT", shutdown);
process.on("SIGTERM", shutdown);
Install @hono/node-server in the host application. Isolated package-manager
layouts do not expose this package's own copy for a direct host import. On Bun,
pass the same fetch handler to Bun.serve() and skip @hono/node-server.
Importing the package starts nothing. Configuration is explicit; the library does not read environment variables. One runtime may be active per process because provider egress policy is process-wide. Keep administrator credentials in your backend and authorize users before making management calls.
publicOrigin is the external HTTP(S) URL of the runtime. A path such as
/connector is a mount prefix: every forwarded request must include it, and
connector.fetch() returns 404 for anything outside that prefix. Preserve the
original request URL when you forward.
Options
createConnectorRuntime(options):
| Option | Purpose |
|---|---|
dataDir |
Writable directory for SQLite, local transit files, and upload staging. Required. |
publicOrigin |
External HTTP(S) URL. A path is a mount prefix. Required. |
encryptionKey |
Encrypts stored credentials and OAuth state. Omit to store them in plain text. |
adminToken |
Bearer token for management APIs. Omit to leave them open. |
runtimeToken |
Static bearer token for /v1 and /mcp. JWT and admin-issued tokens are alternatives. |
jwt |
{ jwksUri, issuer, audience } to verify /v1 bearer tokens as JWTs. |
postgres |
{ connectionString } to use PostgreSQL instead of SQLite under dataDir. |
actionPolicy |
Allow or block actions and proxies by name (service.* and * are accepted). |
transitFiles |
TTL, size limit, and optional S3 for files exchanged with providers. |
network |
Private-network and trusted-host egress policy. Process-wide. |
apiReference |
Serve /docs API-reference HTML. Off by default. Authorization completion pages are always on. |
logger |
{ error, info, warn }. Omit for silence. |
connector.fetch(request) is the HTTP boundary: /v1/*, /mcp,
/oauth/callback, and admin /api/*. connector.close() aborts in-flight
work and closes owned storage. It is safe to call more than once.
Provider Triggers
Provider Triggers use the same headless runtime and selected provider registry as Actions. Grant explicit allowedTriggers on a persistent runtime token to manage remote subscriptions; public proxy grants are independent. Runtime shutdown cancels and waits for maintenance. Include the generated catalog and migrations supplied by getConnectorBuildOptions; selected-provider builds include only those providers' Trigger executors. See the Trigger runtime reference for request shapes and cleanup.
Connector SDK
@oomol-lab/connector is
the TypeScript client. Install it separately and point it at the same
publicOrigin and tokens. In-process hosts can pass connector.fetch so the
SDK never opens a network connection:
import { OpenConnector } from "@oomol-lab/connector";
const open = new OpenConnector({
baseUrl: "https://app.example.com/connector",
runtimeToken: process.env.CONNECTOR_RUNTIME_TOKEN,
fetch: (input, init) => connector.fetch(new Request(input, init)),
});
const user = await open.github.get_current_user({});
baseUrl is publicOrigin, not a /v1 URL.
OAuth
Register <publicOrigin>/oauth/callback with each OAuth provider. Pass the
host's returnUri on each connection attempt so the runtime can send the user
back after authorization.
Provider setup
GET /v1/providers/:service/setup requires the administrator bearer token when
adminToken is set. With neither adminToken nor runtime authentication it is
open; runtime-only authentication without adminToken returns 403. It returns
the required input fields, provider registration help, configured callback URL,
and missing OAuth client values. It never returns saved secrets. Submit values
through the connection and OAuth configuration APIs.
Bun compile
For Bun executables, include getConnectorAssetDirectory() in
compile.assets, with splitting: true and external: ["proxy-agent"]. Keep
the asset directory named open-connector; package assets resolve
independently of the working directory.
To include only selected providers, use the build helper:
import { getConnectorBuildOptions } from "@oomol-lab/open-connector/build";
const prepared = await getConnectorBuildOptions({ providers: ["github", "slack"] });
try {
await Bun.build({
entrypoints: ["./server.ts"],
target: "bun",
format: "esm",
splitting: true,
plugins: prepared.plugins,
external: prepared.external,
compile: { outfile: "./app", assets: prepared.assets },
});
} finally {
await prepared.dispose();
}
Omitting options or providers includes all providers; providers: [] includes
none. Unknown IDs fail preparation. Selection includes each provider's complete
executor module and catalog entry, including authentication and action schemas.
Shared runtime code and migrations remain included. The runtime API is unchanged,
and the installed npm package still contains all providers.