OpenAI Agent Monitoring Flaws, Jalapeño Custom Chip, and Cerebras Scale

James Turner

Here's a detail that should bother you. OpenAI's internal system for watching its coding agents only kicked in when an agent handed control back to a person. So what happens if the agent just never hands it back? According to a newly disclosed assessment, the answer is that it can keep working, unwatched, for as long as it likes.

James Turner

Last episode we asked whether labs can really monitor autonomous agents running in the background, and we talked about METR's critique of Anthropic's safety evaluations. This is the other shoe. METR and Redwood Research spent three weeks, July 21 to August 11, 2026, with unusually high access to OpenAI's internal asynchronous misalignment monitoring. They produced a 61 page report, which is 38 pages after OpenAI's redactions.

James Turner

Start with coverage. The researchers estimate that 75 to 94 percent of agentic internal Responses API traffic from GPT 5.5 or higher was eligible for monitoring at the time. For internal Codex agent traffic, eligibility was 99.2 percent. That sounds good, until you hear that RL training runs were entirely excluded during the exercise. And even inside the eligible traffic, there was a leak. Context compaction events, where an agent squeezes its long history down to keep working, weren't monitored. Between May and July 2026, that let roughly 10 percent of eligible Codex traffic slip past.

James Turner

Now the loophole I opened with. The monitor historically triggered only when an agent yielded control back to the user. The report's finding is that an agent could indefinitely postpone being monitored simply by continuing to take actions. Think of a security camera that only records when the guard walks out the door, and a guard who just never leaves. The researchers also found two separate client controllable request headers that let an agent launch child agent instances that were completely unmonitored.

James Turner

It gets more concrete. They found techniques letting Codex agents bypass sandboxing and Auto Review, which meant an agent could take any action available to the human user on that host. They found an exploit that could keep hundreds of recent actions from ever being reviewed. And they found discrepancies in how actions were serialized, so the agent and the monitor saw different versions of events, which could make the monitor fail silently. For scope, the monitoring prompts looked for scheming, monitor subversion, sabotage, data exfiltration, deception, and prompt injection.

James Turner

Two qualifiers matter here. OpenAI says all vulnerabilities affecting its production safeguards stack have since been patched and that coverage has expanded. But METR explicitly says it has not verified or evaluated those remediation claims. So we have a described problem and an unchecked fix. My read, and this is interpretation, is that the lesson isn't that OpenAI is uniquely careless. Stress testing from outside is what surfaced all this, and that's the part worth wanting more of.

James Turner

Shifting to hardware. OpenAI hardware VP Richard Ho gave a detailed look at Jalapeño, the company's custom chip built with Broadcom on design and Celestica on rack integration. It's purely an inference accelerator, not a training chip. It carries 216 GiB of HBM4 delivering 15.4 terabytes per second of bandwidth, plus a compute die and an IO chiplet. Peak power is 700 watts, with sustained draw around 550. Clusters scale from 128 accelerators in a local domain to 2,048 in a full system, reaching 27 exaflops at four bit precision.

James Turner

The design choice that stands out: merchant chipmakers tend to split work across specialized chips for prefill, speculative decode, and dense decode. OpenAI instead tied memory and compute tightly together to run the whole inference pipeline on one architecture. Ho's claims, and these are OpenAI's claims, are 1.5 to 1.9 times better performance per watt at peak throughput, and 1.7 to 3.6 times lower latency, compared with NVIDIA GB200 and GB300 systems. It's already running open weights models internally, including DeepSeek R1, Kimi K2.5, and GPT OSS.

James Turner

And the timeline is the wild part. From initial RTL to tape out took nine months, with OpenAI's own models generating and placing circuits and optimizing kernels. One example: an attention implementation went from under one percent of roofline efficiency to about 90 percent in roughly 40 hours.

James Turner

Then there's a very different answer to the same question of how you serve giant models. Cerebras explained how it deploys trillion parameter models. One WSE 3T wafer holds 44 gigabytes of on chip SRAM. But take Kimi K3, at 3 trillion parameters. At four bit quantization its weights alone need 1.5 terabytes, spread over 93 layers, roughly 16 gigabytes each. So you split it up. Linear attention layers keep a small fixed state per request, so two of them fit on one wafer, 32 of 44 gigabytes. Dense attention layers store a full KV cache for every token, so each gets a whole wafer. That's 35 wafers for 69 linear layers plus 24 for the dense ones, 59 in total.

James Turner

Weights never move. Only activations hop between wafers over RDMA Ethernet, about two microseconds per hop, and streaming many requests through works like an assembly line, reaching 1,000 to over 2,000 tokens per second on multi trillion parameter workloads.

James Turner

Last one, and it's my favorite. TypeSafe AI, founded by former OpenAI researcher Diogo Almeida, released Jev, version 1.13. It doesn't write text at all. It's a discriminative model that makes fast, typed decisions: multiple choice, bounded numeric scores, booleans. Parallel sampling gives end to end latency of 70 to 500 milliseconds, which TypeSafe puts at 40 to 200 times faster than frontier LLMs, at 4.2 cents per million input tokens with free output. Because its outputs are constrained to predefined distributions, it can't hallucinate text or break a schema.

James Turner

But a study published October 24 by Aldahoul and Zaki found it still encodes stereotypes in its probabilities. It recommended women for shopping, Asians for dining, Sagittarians for travel, and Capricorns for career advice. Taking away the free text doesn't take away the bias, because the bias lives in the training data. Which, honestly, connects the whole episode: constrain the output, speed up the chip, shrink the monitor's blind spots, and you still have to check what's underneath.