Monday AI Brief #12

This is what takeoff feels like. Anthropic and OpenAI have been explicit about their intention to create an intelligence explosion, and employees at both companies have recently confirmed that their models are significantly accelerating their own development.

This week we’ll talk about what that means, considering the trajectory of future progress, our increasing inability to measure the capabilities and risks of the frontier models, and some ideas for how humanity can successfully navigate what is coming.

Subscribe by email RSS feed Longer version

My writing

A Closer Look at the “Societies of Thought” Paper — A fascinating recent paper argues that reasoning models use internal dialogue to make better decisions. I look at what they found, how they found it, and what it does (and doesn’t) mean.

On Recursive Self-Improvement

The intelligence explosion has begun: AI is meaningfully accelerating its own development. Dean Ball considers what’s happening now and where we’re headed soon.

America’s major frontier AI labs have begun automating large fractions of their research and engineering operations. The pace of this automation will grow during the course of 2026, and within a year or two the effective “workforces” of each frontier lab will grow from the single-digit thousands to tens of thousands, and then hundreds of thousands.[…]

Policymakers would be wise to take especially careful notice of this issue over the coming year or so. But they should also keep the hysterics to a minimum: yes, this really is a thing from science fiction that is happening before our eyes, but that does not mean we should behave theatrically, as an actor in a movie might. Instead, the challenge now is to deal with the legitimately sci-fi issues we face using the comparatively dull idioms of technocratic policymaking.

How close is AI to taking my job?

We have a benchmark crisis: many existing benchmarks are saturated, and it’s hard and expensive to create new evaluations that challenge the frontier models. Epoch’s Anson Ho takes a different approach—instead of creating a formal new benchmark, he asked AI to tackle a couple of his recent work projects. Did they succeed? No, but the nature of their failure is informative.

We are not prepared

Great post from Chris Painter that explains an increasingly serious challenge for AI safety:

My bio says I work on AGI preparedness, so I want to clarify:

We are not prepared.

Over the last year, dangerous capability evaluations have moved into a state where it’s difficult to find any Q&A benchmark that models don’t saturate.

AI manipulation

AI manipulation doesn’t get as much press as biosecurity or cyberwarfare, but there are good reasons to worry about AI manipulating humans. An AI with superhuman persuasion can enable authoritarian rule, cause social chaos, or simply take over the world. AI Policy Perspectives interviews Sasha Brown, Seliem El-Sayed, and Canfer Akbulut about their work studying AI manipulation. Lots of good thoughts about what AI manipulation is, why you should worry about it, and how to study it.

What is the impact of AI on productivity?

How much does AI actually increase worker productivity? And are we seeing evidence of that in economic productivity statistics? Alex Imas looks at the evidence so far.

Here is the summary of the evidence thus far: we now have a growing body of micro studies showing real productivity gains from generative AI. However, the productivity impact of AI has yet to clearly show up in the aggregate data.

How can I communicate better with my mom?

Anthropic would like to remind you that ads in AI could go really badly.