Ezra Klein Interviews Jack Clark, Part 1

Ezra Klein and Jack Clark? ($) Shut up and take my money.

Jack always has interesting thoughts about the larger social impact of AI as well as the trajectory of the frontier models. The whole interview is great, but I want to focus on six topics I found especially interesting and/or surprising:

• Model personality
• Claude’s moral preferences
• Excellent but awkward life advice
• Jobs and employment
• Public policy
• Where we’re headed

1: On the origin of personas

Anthropic pays more attention to the personality and psychology of their models than any other lab. That comes up repeatedly throughout the interview: it’s clearly important to Jack that people understand the significance of model persona. The persona model lets us:

• Generate accurate predictions about LLM behavior, and
• Form effective strategies for shaping that behavior

I’d assumed that personas were largely artifacts of generalizing across the vast amount of human behavior in the training set, but Jack goes further, arguing that a sense of self is a consequence of intense training on reasoning and accomplishing tasks:

to do really hard tasks, these systems seem to need to imagine many different ways that they’d solve the task. And the kind of pressure that we’re putting on them forces them to develop a greater sense of what you or I might call self.

Persona isn’t merely the result of mimicry, but a useful (perhaps even necessary) attribute for agentic behavior. While Jack doesn’t extrapolate further, this suggests that more extensive training toward reasoning and agency might drive a stronger sense of self, and perhaps even some form of consciousness.

2: Claude doesn’t like horrible things

A few months ago, Anthropic began experimenting with letting Claude end conversations it didn’t like. That’s important preparation for engaging with future models that may well be moral patients whose welfare and desires are important. Claude’s choices about what conversations to terminate are telling:

It was conversations that related to extremely egregious descriptions of gore or violence or things to do with child sexualization. Now some of this made sense because it comes from underlying training decisions we’ve made. But some of it seemed broader. The system had developed some aversion to a couple of subjects.

This is consistent with one of the most surprising properties of LLMs: they are very good at moral generalization. A model that has adopted a “good” persona is remarkably good at figuring out how to be good in unexpected situations. (Conversely, if a model infers from its training data that it is supposed to be “bad”, it will generalize equally well to being bad in unexpected ways).

There’s an open question about the target of alignment: do we want obedience or virtue? Claude’s preferences about ending conversations suggest that it is not merely capable of virtue, but actively prefers it when offered a choice.

3: Excellent but awkward life advice

Jack has three observations about how to personally navigate agentic AI that I find especially interesting in combination.

First, a clever spin on using AI to help maximize deep work:

I think most people — at least this has been my experience — can do about two to four hours of genuinely useful creative work a day. After that you are, in my experience, trying to do all the turn-your-brain-off schlep work that surrounds that work. I’ve found that I can just be spending those two to four hours a day on the actual creative hard work. And if I’ve got any of this schlep work, I increasingly delegate it to A.I. systems.

Second, an observation that we are all moving up a level in the org chart:

Everyone becomes a manager, and the thing that is increasingly limited, or the thing that’s going to be the slowest part is having good taste and intuitions about what to do next.

Finally, a reminder to define yourself rather than letting AI define you:

There will be people who have cocreated their personality through a back-and-forth with an A.I., and some of that will just be weird. They will seem a little different from regular people. There will maybe be problems that creep in because of that.

And there will be people who have worked on understanding themselves outside the bubble of technology and then bring that in as context with their interactions.

I think that latter type of person will do better. But ensuring that people do that is actually going to be hard.

This is all excellent advice. But let me summarize in my own words:

To thrive in the new AI world, be a high-agency person. Have a strong sense of self and good taste about what to work on.

He’s absolutely right, of course, and on the margin this is all great advice. The awkward part is that for numerous reasons, many (most?) people are not particularly high agency and don’t have an easy path to becoming so. For some people, AI is a force multiplier for agency and productivity, and that’s fantastic. But for a great many people, there is no clear way to remain useful and employable.

At the risk of pointing out the obvious, this is entirely a coordination problem that a fully functional society could readily solve. Having half the population retain all their previous skills and abilities, and half the population gain new superpowers should be a great problem to have. But here we are.

In Part Two we’ll consider employment, public policy, and where all of this is headed.