Files
Kevin Cui da5a3ceed8 refactor: remove unconsumed and duplicated surfaces (#441)
Removes code that has no consumer and collapses code that existed in
several copies, across the server, the console and the tooling. Every
deletion was traced to zero production consumers (in this repo, oo-cli,
open-flow, connector-sdk and the hosted connector) and checked against
git history before landing. Net about -1.4k lines excluding the
lockfile. Behavior-preserving by construction: no `/v1` or `/api`
request or response field changes, no error code or status changes, no
provider definition or action changes (catalog output is byte-identical
except for one dropped key, see below).

## What goes away

**Core and server**
- `src/core/catalog.ts` `allActions` / `findProvider` / `findAction`
(never had a caller; `CatalogStore.actionsById` owns the lookup).
- `defineBearerProviderProxy` and its definition type (zero callers
since the proxy tests were deleted; its option surface had already
drifted behind `defineProviderProxy`).
- `escapeHtml` in `http-utils.ts`, `IConnectServerOptions.staticRoot`,
`MarketplaceService.isProviderEnabled`, `MailProtocol.searchUids` /
`fetchSummaries`, the constant-typed `peek` / `skipAttachmentBodies`
mail options.
- `ActionPolicyService.evaluate` / `evaluateProxy`: every production
path already goes through a snapshot; the shortcut evaluated only the
deployment layer and silently skipped runtime and token grants.
- The constructor-injected `actionPolicy` on `ActionRunner` /
`ProxyRunner` and the allow-all fallbacks behind it: the `policy`
snapshot on the `run()` input is now the only injection path and is
required, as is `getPolicySnapshot` on `IMcpServerOptions`. An omitted
snapshot can no longer evaluate to allow-all by accident.
- `IMcpServerOptions.providerLoader` / `actionPolicy` (never read).
- `CatalogStore.providerSummaries` (only a test read it; the served JSON
string and ETag are unchanged).
- `scenario` had three producers; `createCatalogStore` is now the only
one. `serializeRuntimeProvider` reads it instead of recomputing, and the
catalog generator stops writing it.
- `google-runtime.ts` hand-rolled timeout replaced with
`createProviderTimeout` / `isAbortLikeError` (message and 502 status
preserved byte for byte), and its `timeoutMs <= 0` bypass is gone
because no caller can pass one (bigquery's `timeoutMs` is an API field,
not this option). `GoogleRequestOptions.timeoutMs` documents the
contract: a positive number of milliseconds, 30 s by default.
- MCP tool name/title/description declared once and shared by
`registerTool` and `/mcp/tools`; a new test pins the projection.
`describeActionCapability` is a pure function of the action, the
snapshot and the connection the caller already resolved, so
`get_action_guide` no longer resolves the same connection three times,
and the no-op `ActionMarkdownContext.providerPermissions` override is
gone.
- `describeSchemaType` / `readSchemaProperties` / `readSchemaRequired`
now live in `src/core/json-schema.ts` with their own tests; the drifted
copies in `mcp.ts` and `action-markdown.ts` are gone, `feishu_app_bot`
uses them too, and `readSchemaProperties` rejects an array-valued
`properties` instead of exposing its indexes as parameter names.
- OpenAPI: the never-populated `RunLog.connectionName` and the
unreferenced `ActionSearchRuntimeResult` alias.
- `action-search`: the never-set `boost` option (inlined) and the
`ActionSearchResult` alias (now `ActionSearchDocument`). `fuzzy` /
`prefix` stay.
- PKCE encoding uses the native `base64url` (2000 random inputs
compared, 0 differences).

**Storage and files**
- Row decoders and the run-log query builder were copied byte-for-byte
into the sqlite, d1 and postgres stores. They now live in
`storage/runtime-sql.ts` (`runtime-store.ts` stays the contract module,
SQL dialects stay per backend): one `listRunLogs(input, maxLimit,
placeholder, execute)` replaces the three list bodies, `readString` and
friends take `RuntimeRow` instead of `unknown` because every driver
already returns object rows, the nine-column `runtime_tokens` list is
`runtimeTokenColumns`, and the SQLite store replaces a disguised
non-null cast with the explicit throw the D1 store already used. A
postgres combined-filter test was added because the `$n` numbering
branch had no coverage.
- Transit file helpers were copied into all four backends.
`assertSafeFileId`, `objectKey`, `metadataKey`, `randomHex`,
`assertFileSize`, `transitFileResponse`, `transitFileRead`,
`uploadResult` and `normalizeDescriptor` now live in
`transit-file-store.ts`, `ITransitFileService` extends the core
`TransitFileStore` instead of re-declaring it, `S3ObjectMetadata` /
`LocalFileMetadata` are gone, the local backend gets a `locate()` for
the stat/expiry prelude, and `src/server/files` no longer imports
`node:crypto`. Stored object keys are unchanged.
- `ITransitFileService.response` is required (all four backends
implement it) and the unreachable buffering fallback is gone.
- The Workers catalog loader drops the pre-#243 `/catalog/apps.json`
fallback. Cloudflare ships code and assets as one version, so a deploy
or rollback cannot reach that state.
- `revoked_at is null` predicates removed from the token stores;
`revoke()` has been a hard delete since #35 and nothing writes the
column. Migrations are untouched on purpose (dropping the column would
break fresh installs).
- `listConnections` / `listConnectionsByService` shared a verbatim
20-line block; extracted, with a test for the marketplace ordering and
`isDefault` flag that had no coverage.
- `TransitFileUpload` / `TransitFileRead` have one owner in
`core/types.ts`.

**Tooling and docs**
- 29 `<x>ActionByName` maps with zero readers, plus the action-name
unions that existed only to type them (`OnlyofficeDocspaceActionName`
and `ClickhouseActionName` included). Catalog output byte-identical.
- `scripts/search-actions.ts`, `scripts/generate-provider.ts` +
`generate:provider` (run `node src/providers/dokploy/generate.ts`
directly), `scripts/export-feishu-permissions.ts` +
`export:feishu-permissions` (it emitted 163 of the 165 scopes the app
actually requests), `dev:api` (identical to `start`), and
`scripts/tsconfig.build.json`. `scripts/typecheck.ts` now has one
project per tsconfig (`src`, `scripts`, `examples`) and checks all three
by default, so `npm run typecheck` is bare; `AGENTS.md` says so.
- `add-provider/SKILL.md` no longer tells agents to commit the
gitignored `action-contracts.generated.ts`; `CONTRIBUTING.md` now names
the same generated files `.gitignore` does.
- `publish-helm.yml` stops emitting the never-read `owner` step output;
the OCI_REPO namespace guard is kept.

**Console**
- UnoCSS + presetIcons + `@iconify-json/logos` (generated an empty
stylesheet since the class-icon path was removed), `providerIconUrl`,
`.provider-icon-css-icon`.
- Unused shadcn scaffolding: `scroll-area`, `separator`, `skeleton`,
`toggle` (its `toggleVariants` folds into `toggle-group.tsx`), the dead
sub-components in card/dialog/select/table/alert, and
`@radix-ui/react-{scroll-area,separator,toggle}`. The trimmed files
carry a header saying they are no longer CLI-regenerable.
- `useChart`, `Metric` / `InfoBlock` /
`firstProviderByConnectionStatus`, ten dead CSS rule families, six theme
tokens with no consumer, 34 orphan locale keys in all six locales, and
three recommended-provider ids plus five `scenarioByService` keys for
providers this repo does not ship.
- `web/src/api.ts` loses the inert `RequestOptions` and collapses onto
`request()` / `send()`; `apiGet` takes the bearer token positionally.

## Things a reviewer should know

- `GET /mcp/tools` description text changes for four tools (they now
match what `tools/list` already served); route, shape and auth tier
unchanged.
- `catalog/apps/<service>.json` and the Cloudflare `apps-NNNN.json`
chunks no longer carry `scenario`. `/v1/providers` and `/api/providers`
bytes and ETag are unchanged.
- Storage row-decode errors lose the backend label ("Expected column x
to be a string."); nothing asserts them.
- A Worker whose assets hold only a pre-#243 `apps.json` now fails to
boot loudly instead of serving a stale catalog.
- KV `create()` now normalizes the stored name and MIME type like R2, S3
and the local backend (trim, `file` / `application/octet-stream`
fallback). Before, KV persisted a whitespace-only name and answered the
upload with it while a later read already returned `file`; now both
agree.
- `googleRequest` no longer has a no-timeout mode: a caller passing a
zero, negative or non-finite `timeoutMs` would now time out immediately
instead of never. No caller does; the option is a positive constant or
the 30 s default everywhere.
- `npm run typecheck` now checks `src`, `scripts` and `examples` by
default (CI already did, via the explicit list that is now gone).
- Also in the diff: `CONTRIBUTING.md` wording, four `scenarioByService`
orphans (`alibabacloud`, `apacheairflow`, `aws`, `snowflake`; matched
with the same compaction the resolver uses, none can ever hit), the
`CatalogAsset` absence union collapse, two extra de-exports in
`action-search.ts`, `SecretJsonTable` kept as a file-local alias in the
SQLite and D1 stores, and four `has-data-[slot=...]` Tailwind conditions
whose emitters were deleted.

## Verification

- `npm run fix-check`, `npx vitest run` (163 files, 1543 tests), `npm
run build --workspace web`, `npm run generate:catalog`, `git diff
--check`, all green on the branch; a residue sweep for every deleted
symbol returns nothing.
- Differential checks beyond the suite: google-runtime timeout behavior
across 8 scenarios (identical to `origin/main`), the run-log query
builder across 2016 inputs in both placeholder dialects and
`listRunLogs` against the inline bodies it replaced over 18 input shapes
x 5 row counts (0 mismatches), MCP tool configs character-identical to
the deleted inline ones, `describeSchemaType` replayed over 60,180
catalog input properties (0 differences), `isManagedFileName` against
the three old regexes over 201,200 names (0 differences), the four
transit backends' upload/read/response outputs for padded and
whitespace-only names, `summarizeProviderConnections` proven a
whitespace-only lift, and an allow-all policy snapshot shown to evaluate
identically to the old undefined path for actions, proxies and
connections.

## Not in this PR

- `googlesheets` / `googlecalendar` / `googlephotos` keep their private
copies of the google request and timeout plumbing that
`google-runtime.ts` owns.
- Three more orphan locale keys (`access.policy.deployment`,
`access.policy.runtime`, `overview.recentFailures`).
- The `TransitFileWriter` alias (35 importers) keeps its name, and
`TransitFileStore.delete` stays on the core contract because callers in
privately synced providers cannot be audited from this repo.
- The explicit `!` non-null assertions on the D1 / Postgres connection
upsert and idempotency claim rows (the SQLite one became an explicit
throw).

---------

Signed-off-by: Kevin Cui <[email protected]>
2026-08-30 23:02:06 -04:00

94 lines
2.6 KiB
TypeScript

import { spawn } from "node:child_process";
import { mkdir } from "node:fs/promises";
import { join } from "node:path";
const rootDir = process.cwd();
const tscBin = join(rootDir, "node_modules/typescript/bin/tsc");
const buildInfoDir = join(rootDir, ".tmp/tsbuildinfo");
const projectConfigs: TypecheckProject[] = [
{ name: "src", path: "src/tsconfig.json" },
{ name: "scripts", path: "scripts/tsconfig.json" },
{ name: "examples", path: "examples/tsconfig.json" },
];
interface TypecheckProject {
name: string;
path: string;
}
interface TypecheckResult {
project: TypecheckProject;
exitCode: number;
output: string;
signal: NodeJS.Signals | null;
}
await mkdir(buildInfoDir, { recursive: true });
const projects = selectProjects(process.argv.slice(2));
const results = await Promise.all(projects.map((project) => typecheckProject(project)));
const failedResults = results.filter((result) => result.exitCode !== 0 || result.signal);
for (const result of failedResults.length > 0 ? results : results.filter((result) => result.output)) {
if (result.output) {
process.stderr.write(result.output);
}
}
if (failedResults.length > 0) {
process.exitCode = 1;
} else {
console.log(`Typechecked ${projects.map((project) => project.name).join(", ")}.`);
}
function typecheckProject(project: TypecheckProject): Promise<TypecheckResult> {
return new Promise((resolve, reject) => {
const outputChunks: Buffer[] = [];
const child = spawn(
process.execPath,
[
tscBin,
"-p",
project.path,
"--incremental",
"--tsBuildInfoFile",
join(buildInfoDir, `${project.name}.tsbuildinfo`),
"--pretty",
"false",
],
{
cwd: rootDir,
stdio: ["ignore", "pipe", "pipe"],
},
);
child.stdout.on("data", (chunk: Buffer) => outputChunks.push(chunk));
child.stderr.on("data", (chunk: Buffer) => outputChunks.push(chunk));
child.on("error", reject);
child.on("close", (exitCode, signal) => {
resolve({
project,
exitCode: exitCode ?? 1,
output: outputChunks.length > 0 ? `\n[${project.name}]\n${Buffer.concat(outputChunks).toString()}` : "",
signal,
});
});
});
}
function selectProjects(names: string[]): TypecheckProject[] {
if (names.length === 0) {
return projectConfigs;
}
const projectsByName = new Map(projectConfigs.map((project) => [project.name, project]));
return names.map((name) => {
const project = projectsByName.get(name);
if (!project) {
throw new Error(`Unknown typecheck project: ${name}.`);
}
return project;
});
}