Agent host: a large shell output crashes the window (renderer OOM), because `AgentHostOutputChannel` rebuilds the whole output on every chunk
- VS Code Version: 1.142.0 (commit 6d62b1d5dace93a33b3a6d3eca1317bd5e15bcb1). The same code is on `main` today. - OS Version: Linux x64 (Debian; workbench launched with `--ozone-platform=headless --disable-gpu`) - Session type: agent host, Copilot CLI (`agent-host-copilotcli`) ## Summary In an agent-host session, one shell tool call that prints 8 MB or more crashes the workbench renderer within about 15 seconds: ``` CodeWindow: renderer process gone (reason: crashed, code: 5) ``` The extension host is torn down, and the pending chat request fails with `Canceled: Canceled … at j_.terminate`. The model is not the problem: it only gets the 8 KB `largeOutput` stub ("Output too large to read at once … Saved to: /tmp/…-copilot-tool-output-….txt"). The renderer still receives the full output and processes it in quadratic time and memory. ## Steps to reproduce 1. Open an agent-host Copilot CLI chat session. 2. Have the agent run a command that prints about 8 MB, for example: `yes 'abcdefghijklmnopqrstuvwxyz0123456789 the quick brown fox jumps over the lazy dog' | head -c 8000000` 3. Within a few seconds the renderer reaches about 4.6 GB and the window crashes. We reproduced this with automation in a headless Linux sandbox: - The real workbench was opened on an agent-host session. - The prompt was submitted with `workbench.action.chat.open`. - A scripted model endpoint made exactly one `bash` call and then waited. The model is a stub, so this doesn't depend on which model is used. The sandbox had 16 GiB of RAM and used at most 5.6 GiB, so the crash is the renderer's V8 heap limit, not system memory. ## Measurements Each row is a single `bash` call. "Allocations" comes from the CDP sampling heap profiler, with GC'd objects included, on the workbench page. | Output size | Result | Renderer RSS peak | Allocated by `String.prototype.replace` in `_acceptState` | Page JS heap peak | |---|---|---|---|---| | 1 MB | OK | 0.8 GB | 83 MB | 109 MB | | 2 MB | OK | — | 376 MB | 387 MB | | 4 MB | OK | 2.4 GB | 1,622 MB (77% of all sampled allocations) | 1.6 GB | | 8 MB | **crash, code 5** | 4.65 GB | — | — | | 14 MB | **crash, code 5** | 4.68 GB | — | — | - Doubling the output multiplies the allocation by about 4.4, which is quadratic. - At 4 MB a forced GC brings the heap from 1,656 MB down to 74 MB, so all of it is short-lived garbage. It is created in one burst while the page's main thread is blocked for about 4 s. - The DOM stays at about 2,000 nodes, so rendering is not the cause. The allocation stack at 4 MB: ``` 1621.6 MB replace this._acceptState(state))); private _acceptState(state: TerminalState): void { this._output = state.content .map(part => part.type === 'command' ? part.output : part.value) .join('') .replace(/\r?\n/g, '\r\n'); ``` `_acceptState` runs on every terminal state change, which means once per streamed `TerminalData` action. Each call materializes and CRLF-normalizes the entire accumulated output. Nothing bounds that accumulated output on the renderer side: - The host trims its own copy (`_trimContent`, `AGENT_HOST_TERMINAL_MAX_CONTENT_LENGTH`) in [`agentHostTerminalManager.ts` `appendOutputTerminalData`](https://github.com/microsoft/vscode/blob/main/src/vs/platform/agentHost/node/agentHostTerminalManager.ts). It still dispatches every untrimmed chunk as `TerminalData`. - The [terminal reducer](https://github.com/microsoft/vscode/blob/main/src/vs/platform/agentHost/common/state/protocol/channels-terminal/reducer.ts) appends each chunk (`tail.output + action.data` / `tail.value + action.data`) and never trims. So N chunks of a total output S allocate about S·N/2 bytes. When an envelope batch is reconciled, all of those `_acceptState` calls run in one synchronous task. Past about 8 MB the transient strings appear to exceed the V8 heap limit before the task finishes, and the renderer aborts. There's a related, smaller cost in the chat path: `appendToolOutput` (`stateToProgressAdapter.ts`) rebuilds `terminalCommandOutput.text = prev + chunk` and runs a full-text normalize/compare per chunk. It wasn't the dominant allocation in our profile. ## Proposed fix Follow `AgentHostPty`: apply streamed `TerminalData` from `onDidApplyAction` instead of rebuilding the output from state on every `onDidChange`. - Append each chunk's `\r?\n` → `\r\n` normalization to `_output`. A `\r` that ends one chunk is carried over, so a CRLF split across chunks still comes out as `\r\n`. - Rebuild from state only for a snapshot (`onDidApplySnapshot`, which also covers replayed buffered actions), a cancelled snapshot refresh, or a non-data action (exit, clear, command start/finish). Reading the delta back out of `state.content` doesn't work. The reducer builds each part with `+`, so it's a rope, and slicing it flattens the whole string again. The change is about 40 lines in `agentHostOutputChannel.ts` and doesn't touch the generated reducer. Results: - **Same output.** A randomized equivalence check against the current `join` + `replace` (600k states: CR/LF split across chunks, command parts, clears, exits, mid-stream snapshots) produced identical output in every case. - **No crash.** We patched the same change into the 1.142.0 bundle and reran the repro: | Output | Before | After | |---|---|---| | 4 MB | page JS heap 1.6 GB | 140 MB; `replace` is no longer a top allocator | | 8 MB | crash, code 5 (renderer 4.65 GB) | completes, renderer 1.0 GB | | 14 MB | crash, code 5 (renderer 4.5–4.7 GB) | completes, renderer 1.4 GB | The remaining growth is xterm's scrollback in the chat mirror. That's a separate, linear cost. ## Related - #337688: an agent host V8 OOM when restoring very large sessions. That one is a different process and a different trigger, but it's the same pattern of unbounded tool or transcript data being held in memory.