multica/e2e/perf

UI performance scenarios

The comment-typing scenario models a long issue thread while three agent runs stream progress. It exists because of MUL-7227, where a summary animation replaced a DOM node per streamed message and typing a comment slowed to a crawl. Component tests could not see it — they run in jsdom, which has no style recalculation, no layout and no main-thread contention.

What it runs against

The real Next.js production build, the real page, the real shell and CSS, the real ProseMirror composer. Only HTTP and the WebSocket are synthetic, so there is no API server, database, daemon, agent or account involved. Nothing disables animations or trims the DOM: the cost being measured lives in that machinery.

It covers the renderer shared by Web and Desktop. Electron's own main-process behaviour is not exercised here.

Running it

Against a frontend you already have running:

pnpm exec playwright test --config=playwright.perf.config.ts
# PLAYWRIGHT_BASE_URL defaults to http://localhost:3000
# PERF_REPORT_PATH=<file> also writes the JSON report

Comparing two refs — this builds and measures each one, in sequence:

node scripts/perf-compare.mjs --base origin/main --head HEAD --out perf-report

The comparison needs Node, pnpm and Chromium. It writes comparison.json and comparison.md. Each side stops its server, waits for it to exit and removes its worktree before the next side starts, and the script exits on its own on success, on failure and on cancellation.

The ordinary e2e suite ignores this directory; it needs a production build and a machine that is not busy doing anything else.

The floating-chat lifetime regression runs against the same production build:

pnpm exec playwright test --config=playwright.perf.config.ts floating-chat-lifetime.spec.ts

It checks that closing removes the message list, keeps the real composer and an in-flight upload alive, shows messages received while closed on reopen, and survives an interrupted exit. It also checks that the message list's ancestor is never scaled. HTTP and WebSocket traffic is synthetic; no backend is needed.

When to run it

Nothing runs this automatically. The routine guard is the component test from #8260, which pins the exact failure mode behind MUL-7227 — a streamed message replacing the summary's DOM node — and runs with the normal suite.

Run the comparison yourself, or through the UI performance report workflow (workflow_dispatch, base and head refs as inputs), when a change touches summary animation, long lists, transcript rendering or global styles — and attach the report to the PR. The cost of doing it by hand instead of on every PR is that a regression outside those areas will not be caught automatically.

What it costs

On a GitHub-hosted ubuntu-latest runner the comparison took 340.5 s: 291.5 s for the two production builds, 26 s for the two measurements, the rest install and startup — plus checkout, dependency and Chromium setup around it. The job is capped at 15 minutes. Locally, a repeat comparison whose builds come out of the turbo cache takes under a minute; the report says when a build was restored.

Reading the result

typing_elapsed_ms is the primary number: how long the user waits from the first keystroke until the editor holds the whole string, main-thread queueing included. recalc_style_ms, layout_ms, task_ms and the long-task counters say where the time went.

Two things are not the same as a fast run, and the scenario reports them as invalid or timeout rather than as a duration: messages that never reached the UI, and a composer that did not end up holding every character. An empty page types very quickly.

One sample per ref. Read a small difference as noise until a second run says otherwise. Locally the spread across three runs of one build was around 1%; on CI, comparing two refs with no product difference between them came out 2% apart on typing and on style recalculation. A threshold needs more CI samples than that before it can be trusted.

Changing the fixture

fixtures/typing-under-live-runs.ts generates every byte the scenario feeds the product, and the report carries the fixture version and a digest of it. Bump FIXTURE_VERSION whenever the generated bytes change — two runs are only comparable when the digests match, which the comparison report states.

Do not add DOM to the page to keep the pressure up. A future virtualization change that legitimately renders fewer nodes should show up as fewer nodes.