Define “Incident”

I watched the state dinner the way I watch every launch event now: muted, on the second monitor, next to a log tail. A long table of tech CEOs, each of whom has at some point told a room of investors that their product changes everything, seated a few chairs away from two men who actually can.

The line that made the rounds came from Xi, standing next to Trump at the White House: “We have both the capability and responsibility to develop and manage AI for good and ensure that development of AI is always under human control.” Everyone nodded. I nodded too. I have been the human in control. It was 3:14 in the morning, I was wearing a hoodie, and my entire authority over the system was a thumb hovering over a button labeled ACK.

The more useful sentence came earlier in the week, from a Treasury secretary rather than a head of state. Scott Bessent said the U.S. proposed “a notification mechanism between the two countries for national security-related AI incidents.” China’s commerce ministry called that week’s talks the two countries’ first dialogue on AI.

A notification mechanism. I know that one. It is a pager.

Every company I have worked at built one, and the building is the easy part: an afternoon of wiring alerts to a phone. Then comes the meeting that never ends, the one about the severity matrix. What counts as a SEV1? Who gets woken up for a SEV2? Is “the model said something strange” an incident, or a Tuesday? I have spent more hours arguing about the definition of an incident than fixing incidents. That is not a complaint. The definition is the policy. The pager is just hardware.

Then there is the failure mode nobody puts in the proposal. Set the threshold low and the thing goes off for every hallucinated citation. By week two, everyone on the rotation has an email filter. By week three, the real alert lands in a folder called Hotline (auto), unread, next to the vendor newsletters. Set it high and it never rings, which is how you end up learning about your incidents from the news.

If you want a preview of how the definition fight goes, look at chips. Washington cleared roughly ten Chinese companies to buy Nvidia’s H200, each allowed up to 75,000 units. Chinese customs then blocked the shipments, and component suppliers paused production. By July, a U.S. trade official put the number that had actually shipped, China and Hong Kong combined, at “very few.” Both sides get to say the door is open. Both sides get to say they are being tough. Where I come from, that is a feature flag set to true in a config file nobody deployed. Nobody is lying. Nothing is shipping.

So here is my prediction, and it is the same one I would make about any new on-call rotation. The line will get installed. It will be tested once, in front of cameras. Then the argument about what it is for will start, and that argument will be the actual policy for the next ten years.

I am not cynical about the phone. I am cynical about the matrix. “Always under human control” is a fine sentence. The next one, which human, paged for what, at which severity, is the one nobody at that dinner has written yet. It is also the only one that matters at 3:14 a.m.

Somewhere, under the orange lights of a bar, another version of me has spent all of September watching two baseball teams politely decline to win a division. He thinks that is the saddest negotiation on television. He did not see this one.

Pager’s on.