Self-hosted Open Flow needs OpenConnector to execute provider Triggers
and own their remote subscriptions. This adds a native Trigger runtime
for 21 Triggers across 18 providers, including polling, webhook
delivery, configuration options, and Feishu resource subscriptions.
## Changes
- Register provider Trigger definitions and lazy executors, expose
permission guidance and the five supported operations (`options`,
`read`, `reconcile`, `receive`, `resource`), and export the catalog
consumed by Open Flow.
- Add independent deployment, runtime, and token Trigger grants.
Existing persistent tokens default to an empty Trigger grant; stateful
operations require a persistent token. Trigger-only grants authorize
registered provider calls through the existing authenticated,
SSRF-guarded transport.
- Resolve `x-oo-connector-app-id` as a stable connection ID for
Triggers, Actions, and public proxy. Conflicting selectors fail, and
unknown IDs cannot fall back to another account. Runtime connection
metadata exposes verified provider account identity.
- Persist subscription ownership, callback nonces, provider state, and
cleanup status using the existing secret codec and SQL leases across
SQLite, PostgreSQL, and D1. Enforce immutable active bindings, validate
callbacks, and share Feishu resource references by provider account
across connection aliases and tokens.
- Protect connections while remote resources still need cleanup, allow
verified same-account reauthorization to recover cleanup credentials and
rebuild released resource bindings with the current connection revision,
and provide administrator subscription inspection, cancellation, and
explicit abandonment.
- Run bounded cleanup after grant or token revocation in Node and
Workers. Open Flow owns public ingress, scheduling, business
checkpoints, deduplication, and active subscription reconciliation.
- Bound Poll event pages to 100 events, preserve provider continuation
checkpoints, and align native Google Drive requests with the registered
proxy. Gmail resumes within a history page before advancing its history
cursor; Google Sheets reads from the next unread row with an inclusive
window that respects the event budget.
- Document the API, permissions, lifecycle, and headless packaging
contracts. Preserve Trigger grants through Web Console token edits.
- Fix the PGlite socket fixture's premature `ReadyForQuery` responses
after extended-query errors, with regression coverage for connection
recovery after both Parse and Execute failures.
## Deployment notes
Apply migrations `0016_trigger_policy` and `0017_trigger_subscriptions`
before starting a PostgreSQL runtime. Grant `allowedTriggers` explicitly
to persistent tokens that manage subscriptions. Native local connections
are supported; SaaS and Marketplace Trigger requests return
`trigger_source_not_supported`.
## Validation
- `npm run fix-check`: passed lint, formatting, and type checks.
- `npm test`: 3435 passed, 20 skipped.
- PostgreSQL suite: 10 consecutive runs passed, 38 tests per run. Both
new protocol regression cases fail against the uncorrected fixture.
- Resource HTTP regressions cover cross-alias/token release and
abandonment, account isolation, and same-account credential rotation.
- Native provider regressions use the registered proxies and drain
250-event backlogs under default, small, and large page settings. All 24
provider contract tests also pass against the compiled modules installed
from the actual tarball. New cases cover empty and populated Sheets
seeds, Gmail history records containing more messages than the event
budget, filtering, deleted messages, transient failures, and JSON
checkpoint persistence. All 11 new cases fail against the preceding
implementation.
- Generated catalog: all 21 Trigger contracts match the Open Flow
consumer catalog.
- Runtime package build and isolated tarball consumption: passed,
including native GitHub subscription creation and cleanup against a fake
upstream.
- Compiled host smoke checks: passed with all providers, GitHub only,
and an empty provider selection.
- Web build and Workers deployment dry run: passed.
- Cross-repository HTTP integration: GitHub and Feishu publish,
callback-to-run delivery, provider account identity, and cleanup after
token revocation passed against fake upstreams.
Real third-party account acceptance remains unverified. Workers
validation covers build and D1 behavior; it does not include a
Cloudflare deployment.
Runtime token authentication prepares the same SQLite SELECT and
last-use UPDATE on every successful request. Reuse those two statements
per `SqliteRuntimeTokenStore` instance while continuing to read current
database rows on every call.
Preparation is lazy: custom migration sources can omit `runtime_tokens`,
so eagerly preparing in the constructor would break existing database
initialization behavior. SQL, parameter binding, row decoding, error
propagation, and public interfaces are unchanged. No token records or
authorization decisions are cached.
### Validation
- `npm run fix-check` passed (lint, formatting, and src/scripts/examples
typechecks). Unrelated formatter changes were excluded from the commit.
- SQLite store and runtime token service suites: **36 tests passed**,
including alternating token bindings, all policy fields, revocation,
independent last-use updates, cross-connection changes, reopening,
custom migrations, and best-effort last-use failure handling.
- `npm test` on Windows: 3,132 passed, 9 failed, 4 skipped. Six failures
were source-scan/catalog timeouts under the default parallelism.
- `npm test -- --maxWorkers=2 --testTimeout=30000`: **3,138 passed, 3
failed, 4 skipped**. Remaining failures are in unchanged provider source
guards: two clone-baseline assertions (Windows backslash paths do not
match the committed slash-separated baseline) and one auth-status
error-code assertion. No provider sources or guard tests are changed by
this PR. The upstream Linux workflow remains the CI check for this
change.
### Benchmark
Run `node scripts/benchmark-sqlite-token-statements.ts` after `npm ci`.
It compares the pre-change store method implementations with the actual
updated store, including row decoding and async method calls. Both use
the same migrated schema, 100 tokens, precomputed inputs, warmup, and
seven alternating measurement rounds. Database creation and seeding are
outside the timed loops; there is no timing assertion in CI.
Windows, Node **24.14.1**, SQLite **3.51.2**, `synchronous=2` (FULL):
| Storage | Lookup/update pairs per round | Baseline median | Reused
median | Baseline / reused per pair | Speedup |
| --- | ---: | ---: | ---: | ---: | ---: |
| In-memory | 30,000 | 1,205.28 ms | 247.22 ms | 40.18 / 8.24 us | 4.88x
|
| File, WAL | 1,000 | 652.60 ms | 591.26 ms | 652.60 / 591.26 us | 1.10x
|
The file-backed case reduces elapsed time by about **9.4%** on this
machine. These are isolated store-operation measurements, not end-to-end
HTTP throughput claims; filesystem synchronization and upstream API
latency affect real workloads.
Co-authored-by: machaoxin0407 <[email protected]>
Co-authored-by: l1shen <[email protected]>
## Summary
Add OOMOL cloud OAuth as a provider authorization source. Users can save
an encrypted project API key, select a cloud provider configuration,
authorize named connections, and run their actions and proxy requests
through SaaS. Existing connections retain their source when the provider
default changes, and third-party credentials remain in SaaS.
- Implement project discovery, OAuth request synchronization, remote
execution, and persistent cleanup of disconnected cloud accounts. Add
SQLite, PostgreSQL, and D1 migrations and storage support, with instance
reset and maintenance tooling.
- Add Console project and authorization settings, localized errors,
disconnect notifications, and styled OAuth completion pages that close
after successful authorization. Align local development callback origins
and proxy routes.
- Improve safe request diagnostics and retry transient project
health-check failures once. Execution requests are not automatically
replayed.
- Group storage implementations by database backend, share guarded HTTP
requests with Marketplace, update Zod to 4.6.5, and load provider icons
from the public catalog.
- Document setup, maintenance, and the manual acceptance checklist.
## Validation
- `npm run fix-check` passed.
- `npm run build:web` passed.
- Latest focused regression run: 346 tests passed across 33 files
covering Console, SaaS project/client/routes, OAuth completion, and
guarded HTTP requests.
- Earlier implementation validation included the broader runtime/storage
suites and a real PostgreSQL integration run. Full deployed
action/proxy, cleanup recovery, and D1 maintenance acceptance remain
documented manual checks; they are not claimed as completed by the unit
tests.
## Deployment notes
Apply migrations `0014_saas_project` and `0015_saas_cleanup_runtime`,
configure encryption and an explicit `OOMOL_CONNECT_ORIGIN`, and enable
the relevant cleanup scheduling. The remote SaaS deployment must support
the project OAuth and remote execution APIs described in
`docs/saas-oauth.md`.
## Summary
The provider browser opened on My connections as soon as a credential
had been configured. Users with a few connections therefore saw only
those providers and could easily mistake that short list for the full
catalog. The list also used viewport breakpoints despite the persistent
sidebar, and its action labels and descriptions limited how many
providers fit on screen.
This change opens on All apps by default and shows counts on both view
tabs. It adds a No setup status filter, makes the displayed count
reflect the number of rendered providers rather than the full filtered
set, and keeps task browsing available in a compact disclosure. Provider
rows are now single click targets with concise status badges.
Container-width breakpoints switch the list between one, two, and three
columns according to the space it actually has.
The Vite API proxy can also target a different local port so the updated
catalog can be previewed without stopping another runtime, and the local
dev script prints the configured API port.
## Verification
- `npm run lint`
- `npm run format`
- `npm run build:web`
- `npx vitest run web/src/providers-page.test.ts web/src/i18n.test.ts
--reporter=dot` (62 passed)
- `npm run fix-check`
- Local preview returned 1,560 providers and displayed three columns at
a 1,440px viewport; the No setup filter showed 18 providers.
Applications embedding Open Connector need its provider APIs and
authorization flows without taking over their HTTP server or shipping
the dashboard. This adds `createConnectorRuntime()` with explicit
configuration, a standard request handler, and shutdown. The existing
standalone application uses the same factory and supplies the dashboard.
`npm run build:runtime` produces an installable Node.js/Bun package
containing JavaScript, types, provider catalog, and migrations. The host
owns the listener and lifecycle, supplies the public URL, and chooses a
return URI for each connection attempt. Existing API contracts and
authorization completion pages are preserved.
One runtime may be active per process because the existing provider
egress configuration is process-wide. The short `docs/headless.md`
covers installation and the host boundary.
This is the shared foundation for #539 (provider setup discovery) and
#540 (npm publication). Both follow-ups depend on this PR and are
independent of each other.
---------
Signed-off-by: Kevin Cui <[email protected]>
Co-authored-by: Kevin Cui <[email protected]>
The Bun single-file executable idled at 378 MiB RSS on macOS (306 MiB
with `OOMOL_CONNECT_CATALOG_LAZY_SCHEMAS=1`) because it parsed all 1463
provider executors at startup, read every embedded catalog file into
memory, evaluated optional dependencies it never used, and kept the
module-graph pages pinned for the process lifetime. This branch removes
each of those costs in its own commit, measured before and after, and
lands the lazy-schema executable at 100 to 167 MiB depending on the
platform, with the default mode roughly halved. No API body, ETag,
status code, error text, SDK/CLI wire shape or Cloudflare Workers
behaviour changes, and no new environment variable is added.
Resident memory after `GET /health` (medians of 3 to 5 alternating
rounds, same harness for both binaries, `main` = d11f0e5f):
| platform | measure | default before | default after | lazy before |
lazy after |
|---|---|---|---|---|---|
| macOS darwin-arm64, native | `ps` RSS | 378.0 MiB | 293.1 MiB | 306.5
MiB | 166.8 MiB |
| Debian 13 linux-x64, native | VmRSS | 343.6 MB | 183.8 MB | 269.8 MB |
99.5 MB |
| Linux arm64, Docker | VmRSS | 325.2 MB | 179.0 MB | 283.1 MB | 109.9
MB |
| Windows 11 x64, native | WorkingSet | 345.1 MiB | 225.5 MiB | 259.5
MiB | 113.6 MiB |
Startup to the first `/health` also drops from about 520 ms to 212 ms on
macOS and from 823 ms to 379 ms on linux-x64. On Linux the peak (VmHWM)
falls as well (370.6 to 305.1 MB default, 301.9 to 197.9 MB lazy), so a
cgroup limit sees the improvement too. macOS `ps` counts 55 to 66 MiB of
freed-but-resident allocator pages and the executable's clean `__TEXT`
pages, which is why the default mode stays above 256 MiB there while the
same binary sits at 184 MB on Linux.
What each commit does, with the macOS delta it was measured with:
- `perf(binary)`: _scripts/build-binary.ts_ passes `splitting: true` to
`Bun.build`, so the 1463 provider executors reached through `import()`
in _src/providers/registry.generated.ts_ become embedded chunks parsed
on first use instead of one 30 MB entry module parsed at startup.
Default 378.7 to 324.2 MiB, lazy 306.6 to 240.8 MiB. The smoke script
now POSTs a no-auth `quickchart` action so every target in the
build-binary matrix proves a chunk loads at runtime.
- `perf(catalog)`: `npm run generate:catalog` also writes
_catalog/apps-index.json_ (the catalog without schemas, 4.9 MB, written
atomically) and the executable embeds it. In lazy mode startup reads
that one file instead of all 1463 provider files, which on the
executable kept 55 MB of embedded pages resident. Lazy 240.1 to 182.5
MiB, default untouched. The index rides on the existing lazy flag, a
missing index falls back to the old scan with a warning, and a stale
index refuses to start with a message naming the fix.
_scripts/ensure-generated.ts_ regenerates when the index is missing or
older than the provider files, so existing checkouts regenerate once.
- `perf(server)`: `@aws-sdk/client-s3`, `pg`, `jose`,
`@modelcontextprotocol/server` with `zod`, and
`@scalar/hono-api-reference` load only when their feature is configured
or first requested, through a one-slot `PromiseCache` (the former
`IsolatePromiseCache`, moved to _src/core/promise-cache.ts_).
`createRuntimeJwtVerifier()` becomes async. Workers keep evaluating them
at app creation through `preloadOptionalServerModules()`, and a new
guard test fails if a static import pins any of them back into the
startup graph. Default 324.9 to 305.0 MiB, lazy 181.6 to 167.1 MiB.
- `perf(catalog)`: `providerSummariesJson` is kept as UTF-8 bytes
instead of a JS string (the catalog contains code points above U+00FF,
so JavaScriptCore stored the 6.5 MB body as 15 MB of UTF-16). The ETag
is still computed from the string, so `W/"62be35-aa331d99"` and the body
bytes are unchanged. Retained JSC heap 26.4 to 17.2 MiB in lazy mode.
- `perf(catalog)`: default mode reads provider files four at a time
instead of `Promise.all` over all 1463, which bounded the parse peak the
allocator kept resident. Default 305.9 to 292.8 MiB, startup +9 ms.
- `perf(server)`: `main()` in _src/server/index.ts_ resolves once
`serve()` has requested the listener and a promise chain owns shutdown,
so the entry module finishes evaluating. Bun issues its one-time
`madvise(MADV_DONTNEED)` over the embedded module graph only at that
point, and the top-level `await main()` had kept it from ever firing.
Linux only: 254.7 to 180.2 MB default and 175.4 to 110.3 MB lazy on the
same binary, RssFile 116 to 41 MB, macOS unchanged.
- `build(binary)` and `chore(binary)`: the smoke script reads the lazy
flag with the server's truthiness rule, asserts the second instance on a
port conflict exits with an address-in-use error, and no longer
hardcodes the MCP tool count. `assertBuildInputs()` refuses to build
when _dist/web/catalog_ (the Workers static chunks written by
_scripts/copy-catalog-assets.ts_) exists, because `compile.assets`
embeds _dist/web_ wholesale and that directory silently added 31 MB to a
binary during this work. Lint ignores are anchored to `/catalog` and the
documented executable size range moves to 150 to 175 MiB (every target
grows by the 4.9 MB index).
Two behaviours of the executable change without a flag, both to match
`node src/server/index.ts`, and both are documented in
_docs/single-binary.md_: a listen failure such as `EADDRINUSE` now exits
with code 1 instead of hanging inside Bun's entry-module wait, and an
uncaught exception thrown while serving now ends the process with exit
code 1 instead of being printed and survived. The other un-flagged
deltas are internal timing: the first use of a provider parses its chunk
(2 to 20 ms once per provider), the first `/mcp` request loads the MCP
module (about 20 to 30 ms once per process), and default-mode startup
pays about 9 ms for the batched reads.
`BUN_FEATURE_FLAG_DISABLE_STANDALONE_MADVISE=1` is Bun's own escape
hatch for the page-release hint.
Not done here, on purpose: pre-compressing `/api/providers` or streaming
`/api/actions` (net zero for the startup metric and both need a flag),
minifying the bundle (about 2 MiB after splitting), and packing the
catalog into a single embedded blob (Bun still touches about one page
per embedded file, roughly 18 MB on macOS and 35 MB on Linux).
Verified on every commit with `npm run fix-check`, `npm test` (184
files, 1789 tests), a darwin-arm64 build and `scripts/smoke-binary.ts`
in default and lazy mode. The final binaries were additionally smoked
natively on Debian 13 x64, Windows 11 x64 and macOS, and `GET
/api/providers` returns the same ETag and byte-identical body on every
platform and mode as the `main` build.
---------
Signed-off-by: Kevin Cui <[email protected]>
Removes code that has no consumer and collapses code that existed in
several copies, across the server, the console and the tooling. Every
deletion was traced to zero production consumers (in this repo, oo-cli,
open-flow, connector-sdk and the hosted connector) and checked against
git history before landing. Net about -1.4k lines excluding the
lockfile. Behavior-preserving by construction: no `/v1` or `/api`
request or response field changes, no error code or status changes, no
provider definition or action changes (catalog output is byte-identical
except for one dropped key, see below).
## What goes away
**Core and server**
- `src/core/catalog.ts` `allActions` / `findProvider` / `findAction`
(never had a caller; `CatalogStore.actionsById` owns the lookup).
- `defineBearerProviderProxy` and its definition type (zero callers
since the proxy tests were deleted; its option surface had already
drifted behind `defineProviderProxy`).
- `escapeHtml` in `http-utils.ts`, `IConnectServerOptions.staticRoot`,
`MarketplaceService.isProviderEnabled`, `MailProtocol.searchUids` /
`fetchSummaries`, the constant-typed `peek` / `skipAttachmentBodies`
mail options.
- `ActionPolicyService.evaluate` / `evaluateProxy`: every production
path already goes through a snapshot; the shortcut evaluated only the
deployment layer and silently skipped runtime and token grants.
- The constructor-injected `actionPolicy` on `ActionRunner` /
`ProxyRunner` and the allow-all fallbacks behind it: the `policy`
snapshot on the `run()` input is now the only injection path and is
required, as is `getPolicySnapshot` on `IMcpServerOptions`. An omitted
snapshot can no longer evaluate to allow-all by accident.
- `IMcpServerOptions.providerLoader` / `actionPolicy` (never read).
- `CatalogStore.providerSummaries` (only a test read it; the served JSON
string and ETag are unchanged).
- `scenario` had three producers; `createCatalogStore` is now the only
one. `serializeRuntimeProvider` reads it instead of recomputing, and the
catalog generator stops writing it.
- `google-runtime.ts` hand-rolled timeout replaced with
`createProviderTimeout` / `isAbortLikeError` (message and 502 status
preserved byte for byte), and its `timeoutMs <= 0` bypass is gone
because no caller can pass one (bigquery's `timeoutMs` is an API field,
not this option). `GoogleRequestOptions.timeoutMs` documents the
contract: a positive number of milliseconds, 30 s by default.
- MCP tool name/title/description declared once and shared by
`registerTool` and `/mcp/tools`; a new test pins the projection.
`describeActionCapability` is a pure function of the action, the
snapshot and the connection the caller already resolved, so
`get_action_guide` no longer resolves the same connection three times,
and the no-op `ActionMarkdownContext.providerPermissions` override is
gone.
- `describeSchemaType` / `readSchemaProperties` / `readSchemaRequired`
now live in `src/core/json-schema.ts` with their own tests; the drifted
copies in `mcp.ts` and `action-markdown.ts` are gone, `feishu_app_bot`
uses them too, and `readSchemaProperties` rejects an array-valued
`properties` instead of exposing its indexes as parameter names.
- OpenAPI: the never-populated `RunLog.connectionName` and the
unreferenced `ActionSearchRuntimeResult` alias.
- `action-search`: the never-set `boost` option (inlined) and the
`ActionSearchResult` alias (now `ActionSearchDocument`). `fuzzy` /
`prefix` stay.
- PKCE encoding uses the native `base64url` (2000 random inputs
compared, 0 differences).
**Storage and files**
- Row decoders and the run-log query builder were copied byte-for-byte
into the sqlite, d1 and postgres stores. They now live in
`storage/runtime-sql.ts` (`runtime-store.ts` stays the contract module,
SQL dialects stay per backend): one `listRunLogs(input, maxLimit,
placeholder, execute)` replaces the three list bodies, `readString` and
friends take `RuntimeRow` instead of `unknown` because every driver
already returns object rows, the nine-column `runtime_tokens` list is
`runtimeTokenColumns`, and the SQLite store replaces a disguised
non-null cast with the explicit throw the D1 store already used. A
postgres combined-filter test was added because the `$n` numbering
branch had no coverage.
- Transit file helpers were copied into all four backends.
`assertSafeFileId`, `objectKey`, `metadataKey`, `randomHex`,
`assertFileSize`, `transitFileResponse`, `transitFileRead`,
`uploadResult` and `normalizeDescriptor` now live in
`transit-file-store.ts`, `ITransitFileService` extends the core
`TransitFileStore` instead of re-declaring it, `S3ObjectMetadata` /
`LocalFileMetadata` are gone, the local backend gets a `locate()` for
the stat/expiry prelude, and `src/server/files` no longer imports
`node:crypto`. Stored object keys are unchanged.
- `ITransitFileService.response` is required (all four backends
implement it) and the unreachable buffering fallback is gone.
- The Workers catalog loader drops the pre-#243 `/catalog/apps.json`
fallback. Cloudflare ships code and assets as one version, so a deploy
or rollback cannot reach that state.
- `revoked_at is null` predicates removed from the token stores;
`revoke()` has been a hard delete since #35 and nothing writes the
column. Migrations are untouched on purpose (dropping the column would
break fresh installs).
- `listConnections` / `listConnectionsByService` shared a verbatim
20-line block; extracted, with a test for the marketplace ordering and
`isDefault` flag that had no coverage.
- `TransitFileUpload` / `TransitFileRead` have one owner in
`core/types.ts`.
**Tooling and docs**
- 29 `<x>ActionByName` maps with zero readers, plus the action-name
unions that existed only to type them (`OnlyofficeDocspaceActionName`
and `ClickhouseActionName` included). Catalog output byte-identical.
- `scripts/search-actions.ts`, `scripts/generate-provider.ts` +
`generate:provider` (run `node src/providers/dokploy/generate.ts`
directly), `scripts/export-feishu-permissions.ts` +
`export:feishu-permissions` (it emitted 163 of the 165 scopes the app
actually requests), `dev:api` (identical to `start`), and
`scripts/tsconfig.build.json`. `scripts/typecheck.ts` now has one
project per tsconfig (`src`, `scripts`, `examples`) and checks all three
by default, so `npm run typecheck` is bare; `AGENTS.md` says so.
- `add-provider/SKILL.md` no longer tells agents to commit the
gitignored `action-contracts.generated.ts`; `CONTRIBUTING.md` now names
the same generated files `.gitignore` does.
- `publish-helm.yml` stops emitting the never-read `owner` step output;
the OCI_REPO namespace guard is kept.
**Console**
- UnoCSS + presetIcons + `@iconify-json/logos` (generated an empty
stylesheet since the class-icon path was removed), `providerIconUrl`,
`.provider-icon-css-icon`.
- Unused shadcn scaffolding: `scroll-area`, `separator`, `skeleton`,
`toggle` (its `toggleVariants` folds into `toggle-group.tsx`), the dead
sub-components in card/dialog/select/table/alert, and
`@radix-ui/react-{scroll-area,separator,toggle}`. The trimmed files
carry a header saying they are no longer CLI-regenerable.
- `useChart`, `Metric` / `InfoBlock` /
`firstProviderByConnectionStatus`, ten dead CSS rule families, six theme
tokens with no consumer, 34 orphan locale keys in all six locales, and
three recommended-provider ids plus five `scenarioByService` keys for
providers this repo does not ship.
- `web/src/api.ts` loses the inert `RequestOptions` and collapses onto
`request()` / `send()`; `apiGet` takes the bearer token positionally.
## Things a reviewer should know
- `GET /mcp/tools` description text changes for four tools (they now
match what `tools/list` already served); route, shape and auth tier
unchanged.
- `catalog/apps/<service>.json` and the Cloudflare `apps-NNNN.json`
chunks no longer carry `scenario`. `/v1/providers` and `/api/providers`
bytes and ETag are unchanged.
- Storage row-decode errors lose the backend label ("Expected column x
to be a string."); nothing asserts them.
- A Worker whose assets hold only a pre-#243 `apps.json` now fails to
boot loudly instead of serving a stale catalog.
- KV `create()` now normalizes the stored name and MIME type like R2, S3
and the local backend (trim, `file` / `application/octet-stream`
fallback). Before, KV persisted a whitespace-only name and answered the
upload with it while a later read already returned `file`; now both
agree.
- `googleRequest` no longer has a no-timeout mode: a caller passing a
zero, negative or non-finite `timeoutMs` would now time out immediately
instead of never. No caller does; the option is a positive constant or
the 30 s default everywhere.
- `npm run typecheck` now checks `src`, `scripts` and `examples` by
default (CI already did, via the explicit list that is now gone).
- Also in the diff: `CONTRIBUTING.md` wording, four `scenarioByService`
orphans (`alibabacloud`, `apacheairflow`, `aws`, `snowflake`; matched
with the same compaction the resolver uses, none can ever hit), the
`CatalogAsset` absence union collapse, two extra de-exports in
`action-search.ts`, `SecretJsonTable` kept as a file-local alias in the
SQLite and D1 stores, and four `has-data-[slot=...]` Tailwind conditions
whose emitters were deleted.
## Verification
- `npm run fix-check`, `npx vitest run` (163 files, 1543 tests), `npm
run build --workspace web`, `npm run generate:catalog`, `git diff
--check`, all green on the branch; a residue sweep for every deleted
symbol returns nothing.
- Differential checks beyond the suite: google-runtime timeout behavior
across 8 scenarios (identical to `origin/main`), the run-log query
builder across 2016 inputs in both placeholder dialects and
`listRunLogs` against the inline bodies it replaced over 18 input shapes
x 5 row counts (0 mismatches), MCP tool configs character-identical to
the deleted inline ones, `describeSchemaType` replayed over 60,180
catalog input properties (0 differences), `isManagedFileName` against
the three old regexes over 201,200 names (0 differences), the four
transit backends' upload/read/response outputs for padded and
whitespace-only names, `summarizeProviderConnections` proven a
whitespace-only lift, and an allow-all policy snapshot shown to evaluate
identically to the old undefined path for actions, proxies and
connections.
## Not in this PR
- `googlesheets` / `googlecalendar` / `googlephotos` keep their private
copies of the google request and timeout plumbing that
`google-runtime.ts` owns.
- Three more orphan locale keys (`access.policy.deployment`,
`access.policy.runtime`, `overview.recentFailures`).
- The `TransitFileWriter` alias (35 importers) keeps its name, and
`TransitFileStore.delete` stays on the core contract because callers in
privately synced providers cannot be audited from this repo.
- The explicit `!` non-null assertions on the D1 / Postgres connection
upsert and idempotency claim rows (the SQLite one became an explicit
throw).
---------
Signed-off-by: Kevin Cui <[email protected]>
## Summary
The local Providers catalog only exposed detailed source categories,
which makes a catalog of more than 1,400 services difficult to browse
when a user knows the task they want to accomplish but not the provider
name. This change adds a stable task-oriented scenario to every
generated catalog provider and turns the local Console into a management
and discovery experience similar to the hosted Console.
The hosted and open-source catalogs are not identical, so the
implementation keeps the hosted Console's explicit classification for
shared SaaS providers while retaining a useful category and metadata
fallback for open-source-only and long-tail services. Existing source
categories remain available as the detailed technical filter.
## Changes
- Add a shared provider scenario resolver with AI, e-commerce,
communication, docs, productivity, marketing, data, developer, and
fallback scenarios.
- Generate and serialize `scenario` in the local catalog and the
`/v1/providers` Runtime response, including OpenAPI metadata and
scenario-aware search.
- Keep shared hosted-Console provider overrides aligned across
normalized underscore, hyphen, and compact service identifiers.
- Split the Providers page into **My connections** and **Discover apps**
tabs.
- Add an eight-scenario task discovery grid with counts, representative
provider icons, scenario filtering, and a resettable filter chip.
- Preserve connection status filters, source category filtering,
provider pagination, provider detail routes, and all existing credential
flows.
- Align the discovery tabs and spacing with the hosted Console's
line-tab treatment and separation between the scenario grid and catalog
panel.
- Add translations for every supported Console locale and regression
coverage for catalog serialization, provider classification, page
defaults, discovery rendering, and locale parity.
## Validation
- `npm run lint`
- `npm run fix-check`
- `npm run build --workspace web`
- `npx vitest run src/core/provider-scenarios.test.ts
src/catalog-store.test.ts web/src/providers-page.test.ts
web/src/i18n.test.ts`
- `npm run generate:catalog`
- Manual local-Console verification at
`http://127.0.0.1:5174/providers`, including scenario selection and
narrow-screen layout.
## Summary
- replace the monolithic Cloudflare `/catalog/apps.json` build output
with a versioned `/catalog/index.json`
- write deterministic, sequential catalog chunks capped at 4 MiB by
their final UTF-8 byte size
- load all indexed chunks through the existing `ASSETS` binding while
preserving `CatalogStore` and public API response shapes
- retain a loader fallback to `/catalog/apps.json` for deployments using
the previous asset format
- request catalog assets as JSON so a missing index is not treated as an
SPA navigation
This is the catalog follow-up requested in #235 after the executable
registry reduction merged in #237.
## Why
On the current `main` catalog (1,210 providers and 12,894 actions), the
generated `apps.json` is 26,030,332 bytes (24.8245 MiB), leaving only
184,068 bytes before Cloudflare's 25 MiB per-asset limit. The provider
catalog already consists of individual source files, so combining
everything into one deployment asset creates an avoidable single-file
limit.
## Cloudflare build comparison
Measured on the same current `main` catalog with Wrangler 4.115.0,
`--dry-run`, and `--minify`:
| Measurement | Before | This PR |
| --- | ---: | ---: |
| Largest catalog asset | 26,030,332 bytes (24.8245 MiB) | 4,192,730
bytes (3.9985 MiB) |
| Catalog asset files | 1 | 8 (1 index + 7 chunks) |
| Total static assets | 11 | 18 |
| Worker minified upload | 11,672.94 KiB | 11,674.91 KiB |
| Worker gzip | 2,805.33 KiB | 2,806.08 KiB |
New builds do not emit the complete `apps.json`. The small Worker
increase is the index validation, compatibility fallback, and chunk
loading logic.
## Compatibility and validation
- provider files are sorted by filename before chunking for
deterministic output
- chunk limits include JSON brackets, commas, trailing newline, and
UTF-8 multibyte characters
- a provider too large for one chunk fails the build with its filename
and byte count
- index version, fields, chunk names, duplicate names, chunk shapes, and
provider count are validated before creating the catalog
- indexed chunks are loaded in parallel and flattened in index order
- a missing `index.json` falls back to the legacy `apps.json`; new
builds emit only the indexed format
- existing executable-service resolution from #237 remains unchanged
Checks:
- `node scripts/generate-catalog.ts` — 1,210 apps / 12,894 actions
- `oxlint . --fix`
- `oxfmt .`
- `node scripts/typecheck.ts src scripts-all examples`
- `vitest run` — 61 test files and 613 tests passed
- Wrangler dry-run builds for both the current baseline and this branch
Closes#235
---------
Co-authored-by: Kevin Cui <[email protected]>
## Summary
- stop generating full executable action ID arrays in the Node and
Cloudflare provider registries
- use the loaded executor module service keys to identify locally
executable providers
- resolve those services to the exact action IDs present in the loaded
catalog before creating `CatalogStore`
- preserve explicit `executableActionIds`, runtime execution metadata,
and public response shapes
This is the first of the two changes discussed in #235. Catalog asset
chunking will follow in a separate PR.
## Why
The generated Cloudflare registry repeated all 12,743 executable action
IDs even though each included provider already has a lazy executor
module entry. The existing generator marks every action from an included
provider executable, so the executor module service keys carry the same
information more compactly.
## Wrangler size comparison
Measured from `v1.3.3` and this branch with Wrangler 4.115.0, the same
Cloudflare bindings/configuration, `--dry-run`, and `--minify`:
| Build | Minified upload | Gzip |
| --- | ---: | ---: |
| `v1.3.3` baseline | 11,935.15 KiB | 2,854.70 KiB |
| This PR | 11,566.36 KiB | 2,777.99 KiB |
| Reduction | **368.79 KiB** | **76.71 KiB** |
The emitted `cloudflare.js` decreased from 12,221,596 to 11,843,955
bytes, a reduction of 377,641 bytes.
## Validation
- generated both provider registries for 1,194 total / 1,192 Cloudflare
providers
- `oxlint . --fix`
- `oxfmt .`
- `node scripts/typecheck.ts src scripts-all examples`
- `vitest run` — 61 test files and 591 tests passed
- Wrangler dry-run builds for both baseline and optimized revisions
Refs #235
## Summary
- add exact coverage for all 524 tools committed in
`Dokploy/mcp@db18449eafdfc8dbd438d392b95c46292069c658`
- expose a static, searchable action catalog split across 48 Dokploy
domains, backed by one lazy shared runtime
- preserve Dokploy MCP tool names and request schemas, with 186 GET
actions and 338 POST actions
- support the two multipart tools through OpenConnector transit files:
- `application-dropDeployment`
- `docker-uploadFileToContainer`
- keep `x-api-key` authentication, public-instance URL validation,
request timeouts, bounded responses, and redacted error details
## Scope
This PR targets the generated tool catalog committed in the pinned
Dokploy MCP revision above. It does **not** claim parity with Dokploy's
newer repository-root OpenAPI document, which currently contains
additional endpoints that are not yet part of that MCP catalog.
All 524 actions are statically defined and locally executable; this does
not add an unrestricted API proxy. Operations that can remove, stop, or
otherwise disrupt resources receive an explicit warning when the
upstream specification does not provide a description.
## Reproducible generation
`npm run generate:provider -- dokploy` dispatches to the provider-owned
generator, which downloads the pinned generated OpenAPI document and
verifies SHA-256
`225972ade1e545cc9a44638e4ff32d73a23ea48298617ca3a37fb5cfff6a058e`
before generating and formatting the 48 domain modules. Additional
arguments are forwarded only to that generator. An optional local source
is accepted only when it has the same digest. The generation pipeline
was run twice with identical output.
## Generated-schema exception
The repository guideline normally prefers hand-authored `s` schema
helpers for provider definitions. This PR requests a deliberate
large-provider exception, following the existing generated-provider
precedent used by integrations such as UnifAPI and Postman:
- 524 upstream schemas are too large and change-prone to maintain safely
by hand
- the source is pinned by commit and content hash
- generated output is split by domain instead of placed in one
monolithic file
- an independent 524-name snapshot test detects missing, duplicate, or
renamed MCP tools
- parity tests also enforce 48 tags, action/handler coverage,
representative schema constraints, and exactly two multipart operations
## Runtime behavior
- maps path, repeated query, JSON-body, and multipart fields from static
operation metadata
- preserves successful token and SSH-key responses; only error details
are recursively redacted
- limits response bodies to 10 MiB and error messages to 16 KiB
- preserves upstream error statuses, including bounded plain-text error
responses
- requires a public HTTP(S) Dokploy instance URL and rejects embedded
credentials and private/reserved targets
## Validation
- Dokploy generation pipeline: 524 actions / 48 domains / 0 unsupported;
repeated run produced identical output
- provider registry and catalog generation: 1,064 apps / 10,768 actions
- TypeScript checks: `src`, `scripts-all`, and `examples` passed
- lint and formatting checks passed
- full test suite: 38 files / 246 tests passed
* feat: enhance localization support across multiple pages and components
- Integrated translation functionality using `useTranslate` in ProvidersPage, ResourcesPage, and RunsPage.
- Updated UI elements to utilize localized strings for better user experience.
- Refactored connection and OAuth client forms to support dynamic language switching.
- Improved styling for OAuth client settings and language selection components.
- Added observability settings in wrangler configuration for better monitoring.
* fix by review
l1shen