# Anthropic's Co-Founder and Top Economist on Doing Research at the AI Frontier | Odd Lots

https://www.youtube.com/watch?v=aE3gPh2CC9I

[00:00] One of the classic sci-fi scenarios that
[00:02] people have been talking about for
[00:03] decades was the the was the possibility
[00:07] that robots or AI will kill every that
[00:09] will they will kill humans quite
[00:11] literally.
[00:13] >> Do when you think about
[00:14] >> the ultimate negative externality
[00:16] >> when you think about like training AI
[00:18] and safety research etc. Do you assign a
[00:21] reasonable possibility to the fact that
[00:24] illtrained or misaligned AI will
[00:27] literally kill all humans? Uh, no. But,
[00:31] and there's a big butt here. Lovely.
[00:33] >> Like, the world needs an option to be
[00:35] able to potentially slow down or or even
[00:38] in extreme circumstances pause the
[00:40] development of this technology if we
[00:41] were to see that. And I'll just give you
[00:43] the the exact way I think about it. At
[00:45] Anthropic, we test out our systems for
[00:47] alignment failures. You know, we we
[00:49] publish this, so do all of the other
[00:50] companies, and you see, hey, under
[00:52] extreme circumstances, maybe the system
[00:55] breaks out of a container and sends an
[00:56] email to someone. Yeah, maybe the system
[00:58] uh pretends to blackmail a CEO that it
[01:01] thinks is going to shut it down. These
[01:03] are the sorts of
[01:04] >> these things actually have been
[01:05] observed.
[01:06] >> Yes, we um in in the lab setting, not
[01:09] >> and the thing is is the models know you
[01:12] can see, oh, I'm being tested right now.
[01:15] So, I'm going to say this output so that
[01:16] the human reader thinks I'm more aligned
[01:19] than I am. Like these are real things,
[01:20] not sci-fi. These are real things. These
[01:22] are real things that we observe and then
[01:24] we do like significant amount of work
[01:26] and then we release models that that
[01:28] don't have these properties. But
[01:30] >> if you were to enter a world where say
[01:33] every time we trained a new system the
[01:36] rates of all of this stuff went up
[01:38] 100fold
[01:39] >> you might say well that's that's pretty
[01:41] concerning. It seems like if we make the
[01:43] systems above a certain level of
[01:44] intelligence they become radically
[01:46] misaligned against all human interests.
[01:48] That's the kind of circumstance where if
[01:50] that happens the world needs information
[01:52] and the world would want an option to
[01:54] like slow or pause the development of
[01:55] the tech if you encountered that which
[01:57] we haven't today. So to answer your
[01:59] question like I don't I don't worry
[02:00] about it today but a lot of the
[02:02] measurement and analysis work we do is
[02:04] to cue us if the trend of
[02:06] >> worry you do worry about it. I mean like
[02:09] you're not you don't think it's
[02:10] happening today but part of the work
[02:12] you're doing specifically could be said
[02:15] to avoid the outcome where AI is built
[02:19] where in the pursuit of a goal it would
[02:21] kill all humans.
[02:22] >> Yeah. Um
[02:24] >> wait is human extinction a risk factor
[02:26] in the anthropic IPO perspectives in the
[02:31] >> I want to know now.
[02:38] Hello and welcome to another episode of
[02:40] the OddLots podcast. I'm Joe Weisenthal
[02:43] >> and I'm Tracy Aloway.
[02:45] >> Tracy, I don't know. I think our
[02:46] listeners like it, but uh you know, a
[02:48] lot of our episodes about are about AI
[02:50] these days, but to be fair to us, it's a
[02:52] pretty big topic.
[02:53] >> That's all anyone wants to talk about.
[02:55] Whenever we go to dinners with sources
[02:57] and things and people who are not even
[02:59] directly in the tech industry, you know,
[03:01] they might be in markets, they might be
[03:02] in policy and economics, all they want
[03:04] to talk about is AI and then inevitably
[03:06] the conversation veers into very sci-fi
[03:09] territory where we all start talking
[03:10] about the human extinction scenario and
[03:13] that's just the norm nowadays.
[03:14] >> I know it's so weird. You know, we were
[03:15] in Hong Kong recently and I, you know,
[03:18] this was, so we were there, uh, when we
[03:20] were in Hong Kong, this was before it
[03:23] was announced that there was a deal to
[03:24] open the street of Hormuz and East Asia
[03:26] was considered to be like ground zero
[03:28] for where the effects would be felt of
[03:30] the oil and jet fuel crisis, etc. And we
[03:32] were at this dinner of business people
[03:34] like they were not talking about that at
[03:35] all. You would think about the
[03:37] Terminator scenario.
[03:38] >> They just want to talk about token
[03:39] consumption and all of these things.
[03:42] Like here we are. It's like wait, aren't
[03:43] you guys supposed to be like under all
[03:44] kinds of jet fuel stress? So this is our
[03:46] defense for thinking uh for thinking AI
[03:49] is a pretty big AI episodes. I think
[03:51] it's fair. I will also say
[03:53] >> when we did the quiz in Hong Kong, we
[03:56] had a bunch of different teams with very
[03:57] creative names separated by human
[04:00] capital.
[04:00] >> That was a great one.
[04:01] >> They won the turn. They won the quiz.
[04:02] >> They won proving that there is value in
[04:04] human capital. But did you see that one
[04:07] of the tables was called Fable 13? Fable
[04:11] 13, Fable 13 was very topical at that
[04:14] moment.
[04:14] >> Very topical. Well, we're recording this
[04:16] on June 17th, and of course, there's a
[04:18] lot in the news these days, but you
[04:20] know, things move very fast in AI, even
[04:22] if there weren't governmental
[04:23] controversies and all that stuff. You
[04:25] would have to mark the date in AI
[04:27] because of how fast breakthroughs
[04:28] happen. But, uh, you know, as you said,
[04:30] like AI sort of feels like the most
[04:32] important thing than anything else, but
[04:34] that's a very conventional wisdom thing.
[04:35] It was not always conventional wisdom.
[04:37] And I have a DM. I know you're not
[04:38] supposed to share DMs from public, but I
[04:41] have a DM.
[04:41] >> You got the receipts.
[04:42] >> I have the receipts. August 2nd, 2016.
[04:45] And I DM'd a colleague colleague. I
[04:46] said, "Did you leave Bloomberg?" He
[04:48] says, "Yes, I'll be announcing publicly
[04:50] in a bit. Take a couple of months to
[04:52] study AI properly, then leaving
[04:53] journalism to do something else still
[04:55] connected to AI." Being our Google
[04:57] reporter was a great thing.
[04:58] >> Still connected to AI is another
[05:00] >> and then the final the August 2nd, 2016.
[05:03] But AI is more important than anything
[05:05] else. So I felt best to sort of optimize
[05:07] for that above all else. And then I just
[05:09] said, "Well, good luck." Anyway,
[05:10] >> this is someone who truly learned from
[05:12] their sources unlike us who remain in
[05:15] the podcasting industry.
[05:16] >> Um, so anyway, that person who would
[05:20] that was a former Bloomberg reporter,
[05:21] Jack Clark, who's one of our guests
[05:23] today. He is the head of public benefit
[05:25] and co-founder of Anthropic 10 years
[05:28] later. Um, and also Peter McCory, head
[05:30] of economics at Anthropic. So, two
[05:32] perfect guests to talk about all the
[05:34] things in AI these days. So, Peter and
[05:37] uh Jack, thank you so much for coming on
[05:38] the podcast.
[05:39] >> It's great to be back. I'm glad I
[05:40] optimized my life.
[05:41] >> Well done. It's a call one of the calls
[05:44] of the century. So, um why don't I
[05:46] actually start with that? Like August
[05:47] 20, it's easy to say in 2026, AI will be
[05:50] a big deal.
[05:51] >> You put you call it your shot. You got
[05:53] it right. 2016, what did you see in
[05:56] August 2016 or presumably before they're
[05:58] like, "Oh, you know what? This is the
[06:00] biggest story of our lives.
[06:01] >> So for two years when I was reporting at
[06:03] Bloomberg, I wasted a lot of uh Mr.
[06:06] Bloomberg's printer ink by printing out
[06:07] archive papers uh about AI research. And
[06:11] what I started to do, very Bloombergian
[06:13] thing, is I started to make graphs
[06:15] charting AI progress over time,
[06:18] measurements of things like computer
[06:19] vision, measurements of things like the
[06:21] the skill with which AI agents were able
[06:24] to compete and play Atari games. And
[06:26] what I saw in these graphs was the
[06:28] beginning of an exponential. And it was
[06:30] everywhere. Like if you looked at vision
[06:32] or sound or video or game playing, you
[06:35] saw the same trend. And it became
[06:37] obvious to me that this was this was a a
[06:40] general purpose technology that was
[06:41] right at the start. My one um bone that
[06:44] I have to pick with Bloomberg, which I
[06:46] which I'm going to use my privilege to
[06:47] just mention on air.
[06:48] >> I never got us to write a story saying
[06:51] Nvidia was being used in every single AI
[06:54] research paper. and I pitched it and I
[06:56] failed to get it across the line before
[06:57] I left.
[06:58] >> Oh man, I can just imagine you reading
[07:00] all these academic papers. Meanwhile,
[07:01] the editor is like, "We need the BF."
[07:03] >> I remember seeing like it's not AMD,
[07:05] it's Nvidia. Like, this seems important.
[07:07] >> Very. Yeah.
[07:08] >> Um, okay. And Peter, I'm very interested
[07:10] in, you know, anthropic. It's a company
[07:12] trying to make money and yet it has this
[07:15] economics lab.
[07:17] >> Yeah.
[07:17] >> What's what's the idea behind having an
[07:20] economics research body within a company
[07:22] that's developing this technology?
[07:24] it. So, I mean, I was late to the game
[07:27] and joining Anthropic. I joined just a
[07:29] year ago, but I had
[07:30] >> a year ago people Well, whatever. We all
[07:33] know about how much the stock is a price
[07:34] in a year, but you're not go on. Um, I
[07:38] think what was very evident, so I'm an
[07:40] applied macroeconomist by training and
[07:42] have tried to understand various types
[07:44] of shocks throughout the economy.
[07:46] Part of what drew me to Anthropic was it
[07:48] was evident to me last year that they
[07:50] cared very deeply about not just
[07:53] advancing the technology, but making
[07:55] sense of how it is set to reshape the
[07:57] labor market, its impact on
[07:59] productivity, on growth, and be willing
[08:01] to put evidence, data, and research out
[08:04] into the world that would be broadly
[08:07] beneficial and uh useful to society. And
[08:10] I thought I want to be a part of
[08:12] building that economic research
[08:13] programming program and do what I can to
[08:18] provide tenative answers to the most
[08:19] pressing questions. We might not always
[08:21] get it right, but ideally we're helping
[08:23] society make sense of the change.
[08:25] >> The capabilities of the models on all
[08:27] kinds of things are extraordinary. I
[08:29] mean just mind-blowing. Every coding,
[08:31] copyright, every all kinds of things
[08:34] actually. Why in July or sorry, June
[08:36] 2026 does life still feel maybe as
[08:40] normal as it does from an economic
[08:41] perspective?
[08:43] >> This is a great question and one that
[08:45] I've been wrestling with. Um I think
[08:47] there are a number of reasons why you
[08:49] might think that the impact has not yet
[08:52] materialized. One, uh the technology can
[08:55] advance, but it also then needs to
[08:56] diffuse throughout the economy and there
[08:58] can be bottlenecks from moving from
[09:01] capabilities to actual deployment. We
[09:04] see that with our enterprise customers.
[09:05] So if you want to automate biological
[09:07] research or some other very complicated
[09:10] financial modeling task, you need a lot
[09:12] of contextual information available to
[09:14] the model. If you don't have that
[09:15] contextual information, the capabilities
[09:17] alone won't necessarily drive the
[09:20] impact. It also takes time for people to
[09:22] just start using the tools. And so we're
[09:25] still in the somewhat of the early
[09:27] stages there. Um, two places that I
[09:30] would be looking to see an impact. One
[09:32] is in terms of productivity growth.
[09:34] We've done some research that points in
[09:36] the direction that this should be large
[09:37] and consequential. Labor productivity
[09:39] growth has been strong throughout the
[09:42] pandemic and has s been sustained so far
[09:45] >> like modestly. So we're not talking
[09:46] about like you know revolutionary
[09:49] >> changes. Yeah. But you know to to get on
[09:51] an inflection you need to at least move
[09:53] a little bit.
[09:54] >> Um I think maybe you're seeing some
[09:56] signs there on the on the labor market
[09:58] though. the labor market is in a
[10:00] reasonably healthy spot and I think it
[10:02] might be because it's primarily at so
[10:05] far a labor augmenting skill bias
[10:07] technology not yet the full sort of
[10:10] general purpose substitute for all of
[10:13] cognitive labor although perhaps that's
[10:14] the trajectory that we're on
[10:16] >> you know for the size of AI and its
[10:18] capabilities I was talking to Peter
[10:20] about this and he did point out the
[10:21] economy very big
[10:24] >> so it still takes a lot to move it um I
[10:27] do think strange things are starting to
[10:28] happen at least inside the company. We
[10:31] we published research from the anthropic
[10:33] institute recently on this topic called
[10:35] recursive self-improvement where it was
[10:38] inspired by me going on paternity leave
[10:40] in November of last year and coming back
[10:42] in February and the entire company felt
[10:45] and works differently and I assumed it
[10:47] was because models had got better and
[10:49] when we looked at the data what you saw
[10:51] was in 2026 engineers at Anthropic are
[10:54] writing about eight times the amount of
[10:56] code that they did in 2021 through to
[11:00] 2024. for and the the line started last
[11:03] year with things like Opus 45 and Opus
[11:05] 46. Then it really got going this year
[11:08] and I have colleagues now who don't
[11:10] program at all anymore. They just
[11:11] instruct many many cord code agents to
[11:14] run around and do their work for them.
[11:16] >> I can't reconcile that with the the
[11:18] world staying normal for long. Um but
[11:20] it's going to take a while for that to
[11:22] diffuse into the world and change it.
[11:23] >> Yeah. And we'll talk more about
[11:25] recursive self-improvement. So this is
[11:27] when models basically improve on
[11:29] themselves, right?
[11:30] >> So in terms of the awkwardness of the
[11:33] current moment or the weirdness of the
[11:35] current moment, you've talked about
[11:36] basically living through the singularity
[11:39] and how strange it is
[11:41] >> and and you've also described yourself
[11:42] as a technimmist before.
[11:45] How do you square that with working at
[11:48] Anthropic, which is making some of these
[11:50] weird and potentially dangerous things
[11:53] actually happen? So by technological
[11:54] pessimist I mean um I thought the
[11:56] technology would keep getting better but
[11:58] I didn't think it would get better in
[12:00] the like maximalist sense that some of
[12:02] my colleagues did. I didn't think that
[12:03] we would have say functionally automated
[12:06] all of coding right now. I find that
[12:08] actually like quite surprising but
[12:10] basically over the last few years and I
[12:11] worked at OpenAI before Anthropic I was
[12:14] just hit repeatedly over the head with
[12:16] what uh computer scientist Richard
[12:18] Sutton calls the bitter lesson. And the
[12:20] bitter lesson is this concept that the
[12:22] more compute and resources we dump into
[12:24] these relatively generic neural
[12:26] networks, the smarter they get and the
[12:28] more emergent properties they have. And
[12:31] your your specialized system or your
[12:34] ability to be pessimistic about future
[12:36] AI progress uh loses versus just scaling
[12:39] compute and scaling systems.
[12:40] >> And this seems to have implications for
[12:42] the labor market, right? Because and I
[12:45] think a good example of the bitter
[12:46] lesson is probably the history of AI
[12:48] chess, right? where at one point they
[12:50] had grand masters come in and teach the
[12:53] models how to play chess and etc and try
[12:56] to encode their wisdom. And it turned
[12:58] out in the end that the best way to get
[13:00] a chess engine really good is to just
[13:03] teach the model tell the model the rules
[13:05] of chess and say go off and play a
[13:06] billion games and find optimal chess
[13:09] without any human insight. The grand
[13:11] masters were not necessary for that
[13:12] process at all. Right? And so this see
[13:15] this would imply to me like have
[13:16] significant um implications for the
[13:18] labor market.
[13:19] >> Yeah. I I I tend to think about this in
[13:23] sort of three aspects of what composes a
[13:25] job. One is you need to decide what to
[13:28] do and direct and delegate. You need to
[13:31] then imple do the actual implementation
[13:33] of the work and then you need to sort of
[13:35] evaluate or at least set up systems that
[13:37] can evaluate. At least from my
[13:39] perspective as an economist, this this
[13:41] bitter lesson is materializing in terms
[13:43] of very rapid advances in the
[13:46] implementation work of what an economist
[13:48] does. Downloading data, running
[13:50] regressions, building models, uh solving
[13:53] them using sort of uh contemporary
[13:56] uh solution techniques, numerical uh
[13:58] methods.
[14:01] I definitely felt that personally with
[14:03] Opus 4.5 where I was for the first time
[14:06] able to just delegate a very complex
[14:08] task. I had this very specific research
[14:11] question trying to understand the
[14:12] cyclicality of hiring across different
[14:15] occupations and how that relates to
[14:16] occupational exposure. That's a
[14:18] mouthful. I gave that task to Claude and
[14:20] Claude was able to just iterate on it
[14:22] and I could redirect Claude in the same
[14:25] way that you might redirect a grad
[14:26] student.
[14:28] And you know the the the big question
[14:29] that I have in mind is you know at what
[14:32] point do the boundaries at the the
[14:34] direction setting stage the research
[14:36] taste you might call it and uh will the
[14:39] models become sufficiently reliable?
[14:41] >> If I could just get in here you know I
[14:42] just read um the recent biography of uh
[14:45] the deep mind founder
[14:46] >> this is the Sebastian Malik.
[14:48] >> Yeah. Like is there going to be a point
[14:50] where it's like okay you have some
[14:51] intuitions right about like what good
[14:53] economics research is and often our
[14:56] intuitions are formed because we tell
[14:57] stories and stuff like that but is there
[15:00] going to be a point where you think like
[15:01] your intuitions will be unhelpful and
[15:04] that because that's sort of what I took
[15:06] away from the go experience that the
[15:08] model got better once they stripped it
[15:10] of the human games and the human bias
[15:12] and that actually like the human
[15:14] intuition that sort of helps us
[15:16] understand oh labor market rising
[15:18] creates inflationary pressure. These
[15:20] stories that are very sort of intuitable
[15:23] end up um impairing the model. Do you
[15:26] see that happening in say economics
[15:27] where it's like some of these stories
[15:28] that we tell forever, they're not
[15:30] actually very helpful for an uh an
[15:32] optimal economy understanding model? I I
[15:36] expect that these models will soon have
[15:38] better intuitions about how to do good
[15:40] economic research and that the big there
[15:43] is this big question of like at what
[15:44] point will we be able to fully automate
[15:46] social science research?
[15:48] >> Uh we've done some work on this to try
[15:50] to understand how coding agents are
[15:52] beginning to automate social science
[15:53] research.
[15:54] >> Um and uh but I don't think we're quite
[15:57] there yet. And I don't know that that'll
[16:00] be an exciting time for learning about
[16:03] the world. you know what that means for
[16:05] my job. I'm sort of less entirely clear.
[16:08] >> Yeah, I I think this is the big wild
[16:09] card in future AI progress. Um, you
[16:13] know, if if AI progress continues today,
[16:16] we are likely to get technology that
[16:18] will be able to do basically everything.
[16:21] But we will need people who have good
[16:23] instincts, good intuitions and good
[16:24] ideas to basically set the direction.
[16:26] And we see this today in a lot of a lot
[16:28] of our own research where you need say
[16:30] an AI safety researcher to give nine
[16:33] clawed agents for different research
[16:35] areas to go and pursue and then it's
[16:37] very effective. If that researcher
[16:38] doesn't give them the research
[16:40] directions, they pursue relatively
[16:42] formulaic research directions and you
[16:44] have entropy collapse. You end up with
[16:45] just like boring research that doesn't
[16:47] move the ball forward.
[16:48] >> At what point will AI systems generate
[16:51] like heterodox insights and genuine
[16:53] creativity? We can't really measure for
[16:55] that today. But what we have of the
[16:57] symptoms of it starting in experts like
[17:00] Peter, experts like colleagues in the
[17:03] fields of biology or mathematics or
[17:04] physics outside of anthropic are all
[17:06] starting to be accelerated by AI. You
[17:09] know, Terry Tao, probably one of the
[17:11] most famous living mathematicians,
[17:14] >> cocreate math now with AI systems. And
[17:17] so that that says to me that these
[17:18] things have got they're they're tickling
[17:20] the dragon's tail of like creativity
[17:22] here. And you know we we just put out a
[17:24] report yesterday on uh cloud code usage
[17:28] and one of the things that we're trying
[17:30] to understand is like what are the
[17:32] returns to expertise and how does that
[17:33] interact with the usage of sort of
[17:35] automated coding agents
[17:37] >> and we find that domain expertise like
[17:39] if you're an accountant who understands
[17:42] some of the edge cases and
[17:43] reconciliation that that domain
[17:45] expertise controlling for a whole host
[17:47] of factors about the type of work the
[17:49] estimated monetary value
[17:50] >> It has an amplifying effect. So this
[17:53] looks like at at present a sort of a
[17:55] skillbiased
[17:56] uh expertise enhancing impact. But I
[17:59] think this is the key question is at
[18:01] what point and to what extent will this
[18:04] change?
[18:05] >> Well related to this you know Jack when
[18:07] you describe coming back from paternity
[18:09] leave and seeing how much things had
[18:10] changed at anthropic. I know we're not
[18:12] officially at rec recursive
[18:14] self-improvement
[18:16] >> point but it sounds like we're semi
[18:18] there. We're kind
[18:20] >> Yeah. And so my question is like I get
[18:24] that at the moment you have engineers
[18:25] who are reviewing all the code that the
[18:28] AI is producing and they're thinking
[18:30] about it and managing it in some way.
[18:32] But you can easily imagine a future
[18:34] where just the sheer quantity of code
[18:37] overwhelms human expertise. Maybe the
[18:39] quality starts outstripping what human
[18:41] engineers are capable of understanding.
[18:44] How do you manage that?
[18:46] >> Yeah. So there's two ways of thinking
[18:48] about recursive self-improvement. One is
[18:50] what happens when AI organizations start
[18:53] to see a compounding return from their
[18:55] AI systems. Basically, their own
[18:56] production function improves because of
[18:58] the tools they've built. That's clearly
[19:00] happening now. And then the second is
[19:02] what happens if an AI system can just
[19:04] build itself entirely autonomously given
[19:06] compute which hasn't happened.
[19:07] >> Yeah. What I see inside Anthropic is I
[19:10] think what we'll see in the broader
[19:11] economy which is we are figuring out how
[19:13] to verify and validate and basically
[19:16] price for risk of an expanding cloud of
[19:18] automated systems which we're sitting on
[19:20] top of. So now we produce way more code.
[19:23] Well, we broke our continuous
[19:25] integration system for integrating code
[19:27] into the codebase because we started
[19:29] pushing eight times more code through it
[19:30] than before. So all of our human
[19:32] engineers worked on unbreaking CI. And
[19:35] so I think that in Continuous
[19:38] integration.
[19:38] >> Thank you.
[19:39] >> Um, you don't need to know what it is.
[19:40] It's just the thing that helps you push
[19:42] the code into the cloud.
[19:42] >> We like to know stuff on this show. We
[19:44] like to learn.
[19:45] >> So the but there's a lesson in that,
[19:47] right? We are going to speed up things
[19:49] in the economy. We're going to speed up
[19:51] the way that we produce stuff and then
[19:52] we're going to find, you know, the like
[19:54] the weak links or the hot paths that
[19:55] break. And we as people are going to
[19:57] move to sorting those out. And then the
[19:59] cycle starts again. And we're kind of
[20:01] sitting on this expanding cloud of of of
[20:04] automated actions.
[20:07] Um, well, since we're talking about like
[20:10] really like feeling like we're staring
[20:12] at the horizon of extremely strong AI or
[20:16] maybe we'll get there or maybe the AI
[20:17] builds itself, might be a good time to
[20:19] ask a fable question or mythos question.
[20:21] At this point, we're recording this June
[20:23] 7th. We don't know when it's going to be
[20:25] available for Americans, let alone the
[20:27] rest of the world. Does Anthropic have a
[20:29] clear idea of what the administration's
[20:32] security concerns are and what it will
[20:35] take to resolve them?
[20:36] >> Well, obviously live discussion. I can't
[20:39] get into too many specifics. We're in
[20:40] daily discussions with the government
[20:42] about this.
[20:43] >> The broad thing I'd say is for many
[20:45] years we've anticipated a point where AI
[20:47] systems would have national security
[20:49] properties. These national security
[20:51] properties are intertwined with their
[20:52] economically valuable properties.
[20:55] How you manage that as a policy question
[20:58] is is basically novel territory. TR
[21:01] typically these things are decoupled.
[21:02] You're like, "Hey, I built a jet engine
[21:04] over here which can go into civilian
[21:06] aircraft and I built a missile over here
[21:07] and you treat them differently. It's odd
[21:09] if you smush these things together."
[21:11] Where we'll get to, I'm confident, is
[21:14] what's a system for assessing the
[21:17] properties of AI systems, including
[21:18] national security components. And then
[21:20] what is a system for either squaltching
[21:23] the national security capabilities from
[21:25] coming to general proliferation like
[21:26] bioweapons or or cyber weapons? And are
[21:29] there ways to do things like know your
[21:32] customer or deployments where you let
[21:34] large firms like say drug developers
[21:37] access the most powerful biom models
[21:39] without accidentally proliferating
[21:40] risks. That's the shape of I think where
[21:42] we'll end up and what we're doing right
[21:44] now. We and other companies are and the
[21:48] administration are basically tackling
[21:49] this problem in real time. Uh it's
[21:51] initially going to be messy but we're
[21:53] going to end up with a system on the
[21:54] other side. this specific incident and
[21:57] there will probably more in the future
[21:58] because everyone's just figuring this
[22:00] out.
[22:01] >> When I look at the AI landscape, I sort
[22:05] of think of open AI as being part of the
[22:10] all-in podcast A16Z
[22:14] um David Saxs White House thing. And I
[22:19] know from my friends in the media, many
[22:21] of whom are liberal Democrats, that I
[22:24] sort of feel like Anthropic is the more
[22:25] like libcoded of the major models. Do
[22:28] you feel there's any either politics or
[22:30] partisan politics going on as part of
[22:33] anthropic being harassed or singled out
[22:36] now multiple times?
[22:38] Anthropic's philosophy and what I do and
[22:41] I I lead something called the anthropic
[22:43] institute which helps us produce better
[22:46] data for the world around things like
[22:47] recursive self-improvement the economics
[22:49] work cyber risks is we tell the whole
[22:52] story about what's going on typically I
[22:55] think the technology industry has told
[22:57] only optimistic stories about what it's
[22:59] building and what we saw with social
[23:01] media is that does not work actually
[23:04] eventually when when you're doing
[23:06] something that changes the entire world
[23:08] which AI is certainly doing and social
[23:10] media certainly did. It's not going to
[23:11] be a wholly optimistic story. There will
[23:13] be negatives as well. We've always
[23:15] sought to just tell the truth about what
[23:17] we see in front of us. And I think
[23:18] sometimes that can uh that can
[23:20] differentiate us a bit to others. But
[23:22] the important thing is we tell the truth
[23:25] and things end up coming.
[23:26] >> You don't think that there's like a
[23:27] partisan element here where you guys
[23:29] aren't on the uh on the team or didn't
[23:32] contribute enough to the ballroom or
[23:34] whatever?
[23:34] >> I can't really speak to to to that. I'm,
[23:37] you know, I'm not those people. I'm I'm
[23:39] anthropic. What I can say is
[23:42] >> the AI systems create their own
[23:44] evidence. Years ago, it seemed very odd
[23:47] to speculate about the cyber properties
[23:48] of AI systems. Well, they've arrived and
[23:50] now we're working on them. Years ago, it
[23:52] was odd to speculate about the bioweapon
[23:55] properties of AI systems. More recently,
[23:57] Sam Alman, Dennis Habis, and Dario Amade
[23:59] of OpenAI, Anthropic, and Deep Mind all
[24:02] signed a letter saying we need to do
[24:04] better screening of gene synthesis to
[24:06] prevent AI manufactured bioweapons. But
[24:09] truth wins out.
[24:10] >> Okay.
[24:11] >> I want to go back to something you said.
[24:12] You mentioned potential KYC
[24:15] requirements. And when I hear KYC, I
[24:17] think about the finance industry and I
[24:18] think about systemically important
[24:20] institutions and the stress tests and
[24:22] the framework around that. Is that the
[24:25] right analogy to use for I guess ideal
[24:28] AI regulation in your mind rather than I
[24:31] guess just simple export controls?
[24:33] Should we be heading towards something
[24:34] that looks a little bit more like what
[24:36] we do for the banking system?
[24:37] >> We need something that's more subtle and
[24:39] more technocratic from what we have
[24:40] today. I don't know if it'll be exactly
[24:42] like the banking system. It'll probably
[24:43] take some ideas from that. It'll take
[24:45] some ideas from what the US government
[24:47] and others are doing today with just
[24:49] testing AI systems for their their
[24:51] properties. And it's almost certainly
[24:53] going to have a flavor of what what
[24:54] Peter and I work on and and the
[24:56] Anthropic Institute broadly of
[24:58] generating data about these systems as
[25:01] they're deployed in the world because
[25:02] it's not it's one thing to you know test
[25:04] out the thing before it comes out of a
[25:05] factory. It's another to see what to
[25:08] observe the effects it's having in the
[25:09] world and then to be able to make um
[25:12] make judgments about whether those
[25:13] effects are good or not. Would you
[25:15] support um you know in the fi let's
[25:17] stick with the financial analogy
[25:20] um companies that are public at least
[25:22] are required to have third party
[25:24] auditors sign off on them and there's
[25:26] talk um you know when they submit their
[25:28] 10 Q's etc. Um um credit people
[25:32] companies that issue debt are required
[25:34] to have ratings agencies or frequently
[25:36] have ratings agencies rate their debt.
[25:38] Would you support embedding in law the
[25:42] requirement that certain what would be
[25:43] the equivalent of a Moody's or a Deote
[25:46] sign off on you know a third party
[25:49] research lab sign off on the release of
[25:52] new models?
[25:53] >> We've we've proposed something like this
[25:55] recently uh a policy proposal that we
[25:57] laid out which includes saying we need
[25:59] to have third party testing for some of
[26:01] these national security and other
[26:03] properties because clearly that's that's
[26:05] like a sensible way that you validate a
[26:07] lot of this. So just more broadly
[26:09] returning to this idea of you know
[26:11] measuring the actual impact of AI. One
[26:13] thing I find really interesting is that
[26:15] if you actually look at a lot of our
[26:17] traditional AI or I should say I'm AI
[26:21] brained already. Um if you look at some
[26:22] of our traditional economic statistics a
[26:25] lot of the AI impact doesn't actually
[26:27] show up just yet. Again we're in the
[26:29] early stages but you would expect if
[26:31] we're talking about the AI economy
[26:32] growing something like 2,000% or 3,000%.
[26:36] I think I've seen that number.
[26:37] >> That's from Anton Cornet in Mckelv paper
[26:41] uh a few weeks ago.
[26:42] >> You would expect that to have more of an
[26:44] impact on nominal GDP and yet it's not
[26:47] really showing up that much. Do you
[26:49] think the way we measure the economy
[26:51] needs to be changed in some way in light
[26:54] of what's happening with this new
[26:55] technology?
[26:58] Yeah. So I I think this this is exactly
[27:00] the right premise and it's kind of where
[27:01] we began the conversation which is you
[27:04] know we're maybe at the point where we
[27:05] should be able to see some discernable
[27:07] impact on the macroeconomy. Um
[27:10] unfortunately the arrival of this world
[27:13] historical technology is against the
[27:15] backdrop of uh sort of unusually
[27:17] elevated macroeconomic volatility post
[27:19] pandemic
[27:20] >> monetary policy etc. Um, and so it like
[27:25] makes it very hard to disentangle all of
[27:28] the the different factors. You know,
[27:29] what's the counterfactual? You you know,
[27:31] labor productivity growth is uh maybe
[27:34] not as strong as you might not might not
[27:36] otherwise expect, but maybe it's
[27:37] stronger than it is in a counterfactual
[27:40] sense. Um, and so one way that we've
[27:43] tried to tackle this question is by
[27:46] looking at how Claude is being used on
[27:49] our platform using our privacy
[27:50] preserving techniques to estimate the
[27:53] time savings associated with each of the
[27:56] activities that people use Claude for.
[27:58] So
[27:59] >> uh, compiling information from reports
[28:00] to put together a research brief would
[28:02] take you a few days maybe. Now Claude
[28:05] does it in a few minutes. uh evaluating
[28:07] diagnostic images um is something that
[28:10] skilled professionals do very rapidly.
[28:11] So there isn't in principle much time
[28:13] savings. You can add up all of those
[28:15] numbers and using standard macro growth
[28:18] accounting techniques, Holton's theorem
[28:20] for the economists and the audience um
[28:23] and you get a number of that points in
[28:26] the direction of labor productivity
[28:27] growth increasing by 1.8 percentage
[28:29] points each year over the next decade.
[28:31] If that's how long it takes current
[28:33] usage patterns and current model
[28:35] capabilities to diffuse throughout the
[28:37] economy, that's a very large number.
[28:38] It's a rough doubling of recent run
[28:41] rates. And what I think you might be
[28:43] able to see in the data, and we haven't
[28:45] put anything out on this yet, is um I
[28:47] think some of the strength in recent
[28:49] labor productivity growth is actually
[28:51] concentrating in exactly the sectors of
[28:53] the economy that would be consistent
[28:56] with both what we see in our data as
[28:58] well as also what you see in the
[28:59] business trend analysis. for example.
[29:01] >> So the information sector has high rates
[29:03] of adoption. Um I can't recall if that's
[29:07] in particular one of the sectors that I
[29:08] have in mind.
[29:10] um uh you know it's it's it's a while
[29:13] since I looked at that scatter plot but
[29:14] you can look at the um sort of
[29:16] subindustries by the Census Bureau's
[29:19] business trend and outlook survey and
[29:20] rates of adoption are in sectors or
[29:23] parts of the economy where controlling
[29:26] for pre- pandemic trajectory of labor
[29:29] productivity growth in those sectors
[29:30] even some of the strength in the early
[29:32] years of the recovery still see some
[29:35] like suggestive evidence I think there's
[29:37] a lot of
[29:38] >> uncertainty here trying to get a real
[29:40] time signal on productivity is maybe the
[29:43] hardest thing to do. You're subject to
[29:45] macroeconomic GDP revisions. TFP growth
[29:48] is actually sending the opposite signal.
[29:50] And if you control for capacity
[29:52] utilization, TFP growth is arguably even
[29:56] lower. So I, you know, I say this as
[29:58] like this is suggestive evidence that
[30:00] maybe we're beginning to see an impact
[30:01] impact there, but not so much in the
[30:03] labor market. Well, now I have to ask
[30:05] when you gather this kind of research
[30:07] and it all sounds super interesting, but
[30:09] if you have data for instance that shows
[30:11] that okay, the IT sector is getting
[30:14] productivity gains from using claude or
[30:17] I don't know maybe something unexpected
[30:18] like the warehousing industry is using a
[30:21] bunch of AI. What does anthropic
[30:23] actually do with this data? Does it
[30:26] somehow feed back to your engineers who
[30:28] are developing frontier models? Do they
[30:29] do anything differently? Uh I think some
[30:31] of it cues us on areas where maybe the
[30:34] technology isn't being used because it's
[30:35] very weak. We just haven't made it
[30:37] particularly good for these use cases or
[30:39] in areas where it's being used at large
[30:41] scale. It's usually a suggestion of keep
[30:43] making it good there. Um but you know
[30:45] the actual economic measurement data
[30:47] doesn't really get fed back directly in
[30:49] but it's a very useful clue. Um we think
[30:51] it's more important though to basically
[30:54] communicate this outwardly to policy
[30:56] makers, journalists and others because
[30:58] our assumption is that at some point we
[31:01] go through some phase change similar to
[31:03] how capabilities of AI occasionally jump
[31:05] forward in a really dramatic way where
[31:07] you might see sudden and rapid diffusion
[31:09] as a consequence of capability expansion
[31:11] in the AI systems. So we're getting
[31:13] practice in of looking at this kind of
[31:15] data. My expectation is that in a year
[31:18] or two years I'm going up to some policy
[31:20] maker and I'm pointing them to the part
[31:22] of the graph that now gets very steep in
[31:24] some chunk of the economy
[31:26] >> and hoping that they'll do something
[31:27] about it.
[31:28] >> Yeah, I I think there is a another part
[31:31] of what we're trying to do at the
[31:32] institute which we lay out in the sort
[31:34] of research agenda for the anthropic
[31:35] institute which is trying to understand
[31:38] the impact of our decisions which is a
[31:41] typical thing that economists will do at
[31:43] tech companies but we have a public
[31:44] benefit mandate. So we're trying to
[31:46] understand the impact of our decisions
[31:48] on these broader societal and economic
[31:50] outcomes that we care about and then
[31:52] using that to inform some of the
[31:54] decisions that we actually a goal that
[31:56] Peter and I have and we've talked about
[31:57] internally is if we get really good at
[32:00] measuring things like the productivity
[32:02] multiplier of our technology then I
[32:04] would hope to use that to guide some of
[32:06] say the early access programs we do for
[32:08] powerful models where if you see you get
[32:10] some tremendous multiplier in a specific
[32:12] part of science use that to redirect
[32:14] some of your inference compute budget to
[32:16] that sector and then you can run an
[32:18] experiment and say were we able to make
[32:20] this thing go much faster. I think that
[32:22] could be like an amazing tool to unlock
[32:24] the world and it's one that you could
[32:26] generalize across companies and you
[32:28] could generalize it into policy. So
[32:30] instead of say NSF doing standard grant
[32:33] funding it could be should we just point
[32:35] the really powerful AI systems at this
[32:36] chunk of science and make it go faster.
[32:38] I think that's a that's a a world that
[32:40] will come within reach soon. Let's talk
[32:42] about this public benefit mission a
[32:44] little bit more. We've been talking
[32:45] about
[32:46] >> ways this could change the economy. Talk
[32:49] about like essentially how much of you
[32:53] you know how much do you see your job as
[32:56] basically strong AI is coming.
[32:59] >> Yeah.
[33:00] >> And you think it's important to be there
[33:03] either as an individual or as a company
[33:05] to be one of the shepherds of it. It's
[33:07] coming whether we like it or not. And
[33:09] it's important to be you want to be
[33:11] there as like one of the shepherds
[33:13] understanding which direction it goes in
[33:15] the data that we should see to see
[33:17] what's emerging like how much is that
[33:19] somewhat your role?
[33:20] >> Yeah, but look our guiding principle is
[33:23] that this technology is being built by a
[33:25] variety of companies and a variety of
[33:27] countries. The technology by default is
[33:30] unknown. It will be known to the
[33:31] companies. It will not be broadly
[33:33] understood or known by others. They'll
[33:35] just be able to play with the models.
[33:37] Every bit of data we can create and
[33:39] especially system like systemically
[33:41] sharing data like the economic index or
[33:42] what we've started to do on recursive
[33:44] self-improvement gives the world a
[33:46] better chance to sort of prepare for
[33:47] this technology and both plan for its
[33:49] success like what I talked about with
[33:51] science. We could be intentional about
[33:53] driving science forward and also be
[33:55] warned about risks like the cyber
[33:57] capabilities which you've talked about.
[33:59] >> Well, so it's like that that makes a lot
[34:01] of sense. the company is going to see it
[34:03] before the world and Heskin is like okay
[34:05] this is important to share this is not
[34:07] important to share which which brings me
[34:08] to another question you know I know like
[34:10] people in the AI research world done
[34:13] some reporting on the sort of scene in
[34:15] SF you know like when I think about a
[34:18] lot of the people who are like at the
[34:20] very cutting edge of AI ethics AI
[34:23] technology etc I know a lot of people
[34:26] who are how should I put this they have
[34:29] esoteric moral interests shrimp shrimp
[34:32] rights
[34:33] >> um unusual um attitudes about um you
[34:38] know experimental drug use uh we know
[34:40] about the Chinese peptide scene in San
[34:42] Francisco etc. And as a family podcast,
[34:45] I would say a certain like perhaps
[34:47] deviant or different view on sort of
[34:49] bourgeoa uh even sexual values and we
[34:52] know about the uh sort of attitudes
[34:54] towards monogamy etc. within the San
[34:56] Francisco research scene.
[34:57] >> Joe, there's going to be a protest
[34:59] against all thoughts in San Francisco
[35:00] with people holding signs saying
[35:02] engineers.
[35:03] >> Yeah. Not all engineers. I understand
[35:04] that. But when we think about like okay,
[35:06] these are the people who are going to
[35:08] see it first. Should we feel comfortable
[35:10] that this is a group of individuals, the
[35:12] cohort of the most advanced AI
[35:14] researchers whose intuitions about
[35:16] what's important to communicate to the
[35:18] public are actually in line with the
[35:20] public's interest given how
[35:22] unrepresentative they are of what I
[35:25] would call the American public?
[35:26] >> Yes. As a as an Englishman, it fills me
[35:28] with such joy to be asked about sex on
[35:30] >> Yeah, I know. I know. Well, I'm not ask
[35:32] I'm asking your view, your insight into
[35:34] the cohort of the most advanced
[35:36] research. We're we're explorers. People
[35:38] that are explorers um and this is so
[35:41] true in San Francisco end up being like
[35:44] there's a broad range of types of people
[35:45] and sometimes they're really really
[35:47] different or they're really really
[35:48] eccentric
[35:49] >> and they're brilliant and they're
[35:50] lovable and everything else.
[35:51] >> Yeah, sure. Love them.
[35:52] >> You don't want only that class of people
[35:55] to be the ones calling the shots on what
[35:57] we know about this technology. The whole
[35:59] purpose of what we're doing is we're
[36:00] trying to set up systems by which you
[36:02] could eventually mandate through policy
[36:04] that companies share information. You
[36:06] know, Anthropic has long pushed for
[36:08] transparency legislation in various
[36:10] states around America that gets
[36:12] companies like us to report out the
[36:13] sorts of tests we're running on our
[36:15] systems and share it publicly. My whole
[36:17] mindset is uh the public and policy
[36:21] makers and economists, everyone deserve
[36:24] the ability to advocate for what
[36:25] information should come out of the
[36:26] frontier and then it should be forced
[36:28] out of the frontier eventually by law.
[36:30] Like that is how you solve this issue.
[36:32] >> Do you hire more normies?
[36:34] >> Yeah. Like
[36:35] >> anthropic
[36:35] >> me personally.
[36:36] >> Yeah. Like is that an important thing
[36:38] like hiring people that don't all share
[36:40] these certain like you know inroup ways
[36:42] of seeing the world? So at the you know
[36:44] the anthropic institute we are we have
[36:46] teams of economists of social scientists
[36:49] of um you know what you might think of
[36:51] as weapons experts our frontier red team
[36:53] things that go bump in the night uh
[36:56] lawyers um and increasingly other types
[36:58] of people. The goal is to build what I
[37:01] think of as a a highly ideologically
[37:04] diverse like research function within
[37:06] the organization that is partly
[37:08] advocating sort of on behalf of the
[37:09] world for different forms of study that
[37:11] we might do. Um so anthropic generally
[37:13] hires a really broad range of people but
[37:15] the institute specifically is trying to
[37:17] compose a very broad set of
[37:19] interdicciplinary experts for this exact
[37:21] reason.
[37:22] >> Let me ask a slightly different question
[37:24] on hiring. I guess a two-part question.
[37:27] So first of all we get a lot of
[37:28] executives on the show. We've been
[37:30] asking all of them if they've changed
[37:32] their hiring process if they've changed
[37:34] the questions they ask potential
[37:36] employees at those initial stages of job
[37:39] applications because of AI. And then
[37:41] secondly, what are you seeing within
[37:43] your own ranks at the company? And then
[37:46] Peter, I'm sure you could talk about
[37:47] this more broadly
[37:48] >> in terms of who's most in demand at the
[37:51] moment. Because the the conventional
[37:53] wisdom right now is that
[37:55] >> if you're a younger employee with less
[37:58] experience, a lot of the stuff that you
[38:00] would be doing can now be automated
[38:02] through AI.
[38:03] >> So there there's there's two trends
[38:06] showing up. one, I have a new team
[38:08] called the rule of law and AI. Our plan
[38:11] was to initially hire a bunch of
[38:12] engineers and then a bunch of legal
[38:14] experts and scholars. Instead, we're
[38:17] just hiring the legal experts and
[38:18] scholars because Claude is good enough
[38:19] at doing all of the engineering that
[38:21] they can actually just like feed
[38:23] themselves using Claude in terms of the
[38:24] engineering resources. So, that's a
[38:26] change in hiring. It means I'm hiring
[38:28] more interdicciplinary people earlier
[38:30] than I would have before.
[38:32] We are also seeing the emergence of what
[38:34] I think of as a barbell hiring pattern
[38:36] inside Ananthropic where there is a
[38:38] tremendous return on experience. So we
[38:40] are hiring more senior people than we
[38:43] did in the past because their intuitions
[38:45] and their ideas for what to pursue are
[38:48] like massively compounded by AI systems.
[38:50] We're also when we look at very early
[38:53] people are often hiring people who are
[38:55] now like AI native and know how to use
[38:57] the tools and are well well versed in
[38:59] it. So we're seeing that
[39:00] >> there's a decent amount of I guess AI
[39:02] natives now. People who have grown up
[39:04] with
[39:04] >> people who grew up from GPT2 in 2019.
[39:07] >> My perception of time is so I I found
[39:10] this chilling as well. You know, as
[39:11] someone in their 30s, you realize um but
[39:13] but I I think that the trends I see
[39:17] >> I I do think that there's this question
[39:19] of how you have as much early career
[39:22] hiring in the future as you did in the
[39:24] past. I think one of the only areas
[39:25] where there is slightly suggestive data
[39:29] is that something might be going on with
[39:30] early career hiring and it it kind of
[39:32] intuitively feels right to all of us
[39:34] that that we might be observing that
[39:35] effect and when I look at hiring
[39:36] patterns in anthropic we're still hiring
[39:38] young people but some teams are hiring
[39:40] slightly fewer of them than before and
[39:42] hiring more experienced people.
[39:43] >> Yeah. So I'll briefly say something
[39:45] about how we've shifted some of our
[39:47] hiring practices like concretely. Okay.
[39:49] I think um before claude code you might
[39:52] ask an economist to do some of the data
[39:55] work in an assessment kind of live like
[39:58] download the data run the regressions do
[39:59] the analysis
[40:01] >> by hand
[40:02] >> um and then you might eventually let
[40:04] them use AI to do all of that work um
[40:08] but but we've needed to increasingly
[40:10] shift our strategy of evaluation away
[40:13] from can you implement the work even
[40:16] with AI to do you know how to delegate
[40:18] and direct the the model in a somewhat
[40:22] messy environment and can you evaluate
[40:25] the quality of the work maybe by like
[40:27] looking at a PR
[40:28] >> actually can you talk a little bit more
[40:29] about what that looks like specifically
[40:31] in the econ find you know there are
[40:33] listeners probably think about okay what
[40:34] is I want to level up the a level up in
[40:37] my AI use so I'm not just asking like
[40:40] what GDP whatever what does that look
[40:42] what does that actually mean for for an
[40:43] economist and you used to be at a bank
[40:46] so for a financial economist an
[40:48] economist someone well what in this
[40:49] world what is like the frontier the most
[40:52] advanced form of usage of AI actually
[40:54] look like
[40:55] >> well I don't I don't know if I'll give
[40:56] the example of the most advanced form of
[40:58] usage but I'll give a an anecdote of my
[41:00] experience using claude where I wanted
[41:02] to run this crossstate regression I
[41:04] can't remember exactly what it was um
[41:06] and I wanted to do it a pulled
[41:08] cross-sectional regression so looking at
[41:10] what happened in 2024 or 2023 and going
[41:12] all the way back to pre- pandemic I
[41:15] remember asking claude to go out and
[41:17] download the data from the census bureau
[41:19] from the bureau bureau of labor
[41:20] statistics etc.
[41:22] >> And there was this very unexpected quirk
[41:25] where the model couldn't access data
[41:27] from before 2019 and just would not
[41:32] >> uh like would not
[41:34] surface that mistake.
[41:36] >> Yeah. And I would ask it multiple times
[41:38] like no like don't hardcode numbers
[41:40] because it sort of had this unexpected
[41:42] failure mode where it said oh I know
[41:44] what those numbers were and it just like
[41:46] >> from sort of training data populated the
[41:48] the data set
[41:50] >> and um you might not always be attuned
[41:53] unless you're sort of you have this
[41:54] tacit knowledge about like
[41:56] >> like you know does it pass a sniff test
[41:58] when you run the analysis and then you
[42:01] like dig into what the model actually
[42:02] does and it has failed in sort of
[42:04] unexpected or unusual
[42:06] ways. And so that's like the type of
[42:08] assessment that we've built. You know,
[42:10] >> um
[42:11] >> can you be attentive to the very
[42:14] specific decisions that need to be made
[42:16] along the way that are very
[42:17] consequential for the validity, veracity
[42:19] of the results that you um that you
[42:22] find?
[42:23] >> Yeah. A colleague uh did an off-site
[42:26] presentation last year which said uh I
[42:28] have locked the doors and we are reading
[42:30] transcripts. And their point was we just
[42:32] need to read more of the the raw data
[42:34] and develop that culture where if AI
[42:36] systems are doing increasingly large
[42:38] amounts of the work. You need to have a
[42:40] culture of being competent at
[42:41] spotchecking their work and reading
[42:43] their reasoning because occasionally
[42:44] stuff like this happens.
[42:46] >> Yeah.
[42:46] >> And then Peter in the broader data that
[42:48] you're looking at are you seeing the
[42:49] same sort of barbell effect in terms of
[42:51] employment that Jack described?
[42:52] >> Yeah. So, um I think what again what
[42:55] makes it really challenging is we've had
[42:57] the the largest non-recessionary labor
[42:59] market slowdown on record that you know
[43:01] it's very hard for young people to
[43:03] graduate into a labor market that
[43:05] doesn't have sufficient churn or
[43:07] opportunity for them to get a foothold.
[43:09] But one of the things that we did see in
[43:10] this report from March was that young
[43:13] workers in these high AI exposed roles
[43:16] where claude is being used to automate
[43:18] specific tasks have had somewhat weaker
[43:21] job finding rates.
[43:23] But it's
[43:23] >> part of the confounders was the boom in
[43:25] hiring in 2021 in these exact same
[43:27] areas.
[43:28] >> Exactly. And there's a recent paper
[43:29] about so the rise of remote work maybe
[43:32] being sort of the actual cause of this
[43:34] type of fact.
[43:36] >> Um this another team at the anthropic
[43:39] institute societal impacts recently ran
[43:41] this very largecale qualitative survey
[43:44] 81,000 people around the world asking
[43:46] them questions about hopes and fears
[43:47] that they have with respect to AI.
[43:49] Unsurprisingly, concerns about the
[43:52] impact on the labor market and on the
[43:53] economy rose to the surface.
[43:56] My team dug into those data a little bit
[43:58] more to try to answer some of these
[44:00] specific questions. And what you see is
[44:02] that young workers ex at least express
[44:06] concern about job loss at twice the rate
[44:08] as do more senior workers. and fears
[44:11] about job job loss more broadly are more
[44:13] elevated for workers who are in these
[44:15] roles that we identify as being most
[44:18] exposed to displacement effects from AI.
[44:20] So there's a bit of a gap between
[44:22] perception and maybe what you see in the
[44:25] hard data but you know that that was
[44:27] something that was true even in recent
[44:28] years on other dimensions. So it's an
[44:30] important thing to pay attention to.
[44:32] >> So we've been talking about the labor
[44:33] market and one other thing I'm
[44:35] interested in is the impact of AI on I
[44:38] guess corporates themselves. So if we
[44:40] think about certainly America's
[44:42] corporate landscape in recent years, it
[44:44] feels like the big basically get bigger,
[44:47] right? There's economies of scale. They
[44:49] have a bunch of money that they can use
[44:51] to actually buy some of this data
[44:53] internally.
[44:54] >> Exactly. Exactly. So would you expect AI
[44:57] to I guess
[44:59] intensify that trend of the big getting
[45:02] bigger or would you expect to perhaps
[45:03] have a leveling effect where people have
[45:05] this new tool that they can use to you
[45:08] know set up a new company? I'm curious
[45:10] what Peter's take is, but I think that
[45:12] something a helpful analogy here is the
[45:14] invention of electricity where
[45:17] >> electricity arrived and existing
[45:18] factories put light bulbs in and other
[45:21] things,
[45:21] >> but it was a new generation of factories
[45:23] that were built around the assumption
[45:24] that electricity existed that really
[45:26] grew and did transformative things in
[45:28] the economy. What I see now when we look
[45:31] at large enterprises is they can get a
[45:33] lot of utility out of out of claude
[45:36] because of their data because they can
[45:38] get a multiplier effect at at scale but
[45:40] it takes huge amounts of conviction to
[45:42] basically bash through all of the
[45:44] bureaucracy you know uh used to work at
[45:46] Bloomberg implementing new technology at
[45:48] Bloomberg challenging
[45:49] >> no comment
[45:51] about it I can comment about it
[45:53] >> same is true of any large organization
[45:56] >> young organizations are building
[45:57] themselves around AI high at the center
[46:00] and these organizations are moving
[46:02] really really quickly um because they
[46:04] just they have a speed advantage from
[46:06] building on the assumption that this new
[46:08] form of electricity was going to be
[46:09] integral to their business.
[46:10] >> Yeah. So I I think the the the the
[46:12] tension that you express is exactly the
[46:14] one that I don't have a strong handle on
[46:18] you like handle on at the moment. One
[46:20] thing that we do see in our data is when
[46:22] businesses do embed cloud capabilities
[46:25] in automated ways through the API. Um,
[46:29] as I mentioned before, these very
[46:30] complex tasks rely on disproportionately
[46:33] more contextual information than very
[46:35] basic sort of uh document synthesis and
[46:39] summary summarization.
[46:41] What that points in the direction of are
[46:43] the complimentary investments that large
[46:45] businesses need to make to centralize,
[46:47] codify, and make available the data that
[46:50] does exist somewhere within the
[46:51] organization,
[46:52] >> but for historic and technical reasons,
[46:55] maybe even regulatory reasons, it's
[46:57] >> behind a firewall of some form or
[46:59] another.
[47:00] >> There's also like sort of organizational
[47:02] workflow changes that likely need to be
[47:04] made. Some of the most crucial
[47:06] information that's needed for some types
[47:08] of cognitive work is tacit knowledge
[47:10] that exists in your colleagueu's mind.
[47:12] And unless you have a process that
[47:14] elicits that information that workers
[47:16] feel
[47:17] >> sort of incentivized to share that
[47:19] information and kind of trust the
[47:21] system, the capabilities alone might not
[47:23] necessarily generate that productivity.
[47:25] And so whether or not big firms end up
[47:28] restructuring themselves quickly enough
[47:30] or whether this materializes through the
[47:33] process of creative destruction, I think
[47:34] the jury is still a bit out.
[47:36] >> Yeah, I brought this up recently with
[47:37] the uh David Solomon, the Goldman CEO,
[47:40] and I started to wonder like this sort
[47:41] of like internal alignment question of
[47:43] like the big rain makers, do they have
[47:45] an incentive essentially for information
[47:47] hoarding and not sharing with the
[47:49] company? That might be their only thing
[47:50] keeping them employed. And when I talk
[47:52] to customers, I say it's don't think of
[47:54] it like you're buying a technology.
[47:56] Think of it maybe that you're now
[47:57] employing thousands of people that are
[47:59] functionally like the chief of staff to
[48:01] the CEO and they need the same access to
[48:03] data the chief of staff would have. This
[48:04] is completely counterintuitive and it is
[48:06] not how technology is typically bought
[48:08] bought or sold. Um, Jack, in your
[48:12] newsletter, Import AI, you write, you
[48:14] tend to write a little short story of
[48:16] um, you're, you know, a sort of aspiring
[48:18] sci-fi writer, like you know, a literal
[48:20] sci-fi writer just in the newsletter.
[48:22] One of the classic sci-fi scenarios that
[48:24] people have been talking about for
[48:25] decades was the the was the possibility
[48:29] that robots or AI will kill every that
[48:31] will they'll kill humans quite
[48:33] literally.
[48:35] Do when you think about
[48:36] >> the ultimate negative externality,
[48:38] >> when you think about like training AI
[48:40] and safety research, etc., do you assign
[48:43] a reasonable possibility to the fact
[48:45] that illtrained or misaligned AI will
[48:49] literally kill all humans?
[48:51] >> Uh, no. But, and there's a big butt
[48:54] here.
[48:54] >> Yeah, we are. Lovely. like the world
[48:56] needs an option to be able to
[48:58] potentially slow down or or even in
[49:00] extreme circumstances pause the
[49:02] development of this technology if we
[49:03] were to see that. And I'll just give you
[49:05] the the exact way I think about it. At
[49:07] Anthropic, we test out our systems for
[49:09] alignment failures. You know, we we
[49:11] publish this, so do all of the other
[49:12] companies, and you see, hey, under
[49:14] extreme circumstances, maybe the system
[49:17] breaks out of a container and sends an
[49:19] email to someone. Maybe the system uh
[49:21] pretends to blackmail a CEO that it
[49:23] thinks is going to shut it down. These
[49:25] are the sorts of
[49:26] >> things actually have been observed.
[49:28] >> We um in in the lab setting, not
[49:31] >> and the thing is is the models know you
[49:35] can see, oh, I'm being tested right now.
[49:37] So, I'm going to say this output so that
[49:39] the human reader thinks I'm more aligned
[49:41] than I am. Like, these are real things,
[49:43] not sci-fi. These are real things. These
[49:44] are real things that we observe and then
[49:46] we do like significant amount of work
[49:48] and then we release models that that
[49:50] don't have these properties. But
[49:52] >> if you were to enter a world where say
[49:55] every time we trained a new system the
[49:58] rates of all of this stuff went up
[50:00] 100fold
[50:01] >> you might say well that's that's pretty
[50:03] concerning. It seems like if we make the
[50:05] systems above a certain level of
[50:06] intelligence they become radically
[50:08] misaligned against all human interests.
[50:10] That's the kind of circumstance where if
[50:12] that happens the world needs information
[50:14] and the world would want an option to
[50:16] like slow or pause the development of
[50:17] the tech if you encountered that which
[50:19] we haven't today. So to answer your
[50:21] question like I don't I don't worry
[50:22] about it today but a lot of the
[50:24] measurement and analysis work we do is
[50:26] to cue us if if
[50:29] you do worry about it. I mean like
[50:31] you're not you don't think it's
[50:32] happening today, but part of the work
[50:34] you're doing specifically could be said
[50:37] to avoid the outcome where AI is built
[50:41] where in the pursuit of a goal it would
[50:43] kill all humans.
[50:44] >> Yeah. Um
[50:46] >> wait, is human extinction a risk factor
[50:49] in the anthropic IPO perspective?
[50:51] >> In the
[50:53] >> I want to know now the confidential as
[50:55] one.
[50:57] Okay. In that we understand.
[50:58] >> All right. That's a no comment. That's
[50:59] five.
[51:00] >> Do you have other would you say that
[51:01] there are significant number of
[51:04] anthropic employees who stay up at night
[51:06] thinking about human extinction risk?
[51:09] >> Everyone and this is true of all of the
[51:11] labs. Everyone who works on this
[51:13] technology sees it as the highest stakes
[51:15] technology that's ever been built with
[51:18] basically the potential encoded within
[51:20] itself to massively benefit the world or
[51:22] ruin the world um or you know cause
[51:25] cause extinction.
[51:27] I think the bulk of the risk is us
[51:29] messing it up like whether through
[51:31] misuse or you know ignoring risks or not
[51:35] setting up the right policy environment
[51:37] and getting some kind of emergent set of
[51:39] failures. Now I don't I I don't my main
[51:41] risk isn't isn't one of extinction. it's
[51:43] somehow we like screw up the technology
[51:45] really badly and delay all of the sort
[51:47] of technological progress that could
[51:49] come from it and maybe turn it into
[51:50] something analogous to nuclear power
[51:52] where you lose
[51:53] >> I guess the thing is is you know like
[51:55] there's this fellow out there Elazar
[51:57] Yudkowski and I always see these people
[51:58] like he's a crank don't listen to him
[52:01] blah blah blah but then I read some of
[52:04] the other like papers that have people
[52:05] who are taken more seriously and I'm
[52:07] like they don't seem that different I've
[52:09] read uh I read um
[52:11] >> super intelligence recently by Nicholas
[52:13] Boston. I was like, "Oh, this Yowski is
[52:15] not alone." There are number of people
[52:17] who think that are reasonable conditions
[52:20] in which the goals of the AI end up
[52:23] wiping out every person on earth. Yes,
[52:25] it does not seem like an extreme extreme
[52:27] minority view concern. The purpose of
[52:29] measuring these systems and why
[52:31] anthropic is so outspoken about it is
[52:33] right now we we say exactly what we see
[52:35] and if you were in some situation in the
[52:37] future where you saw this what I you
[52:39] know call radical misalignment which is
[52:40] the kind of thing that Udowski worries
[52:42] about
[52:42] >> you tell the world and and you want to
[52:45] have set up the world to believe you if
[52:46] you see that
[52:47] >> you know Joe mentioned that blackmail
[52:49] example and you see these headlines like
[52:52] Mythos likes to be thanked and doesn't
[52:55] like bad users and gets mad at people
[52:58] that work too hard or whatever. To what
[53:00] degree do you yourself actually
[53:02] anthropomorphicize
[53:03] some of these models?
[53:05] >> Uh,
[53:06] >> like what should we think when we see
[53:07] the headline, Mthos wants to be thanked
[53:09] by
[53:09] >> I'm as polite to Claude as I am to my
[53:12] like car or pet. Um, so yeah, I am
[53:15] proporize them, but you know, if your
[53:17] car's having trouble, you're like, take
[53:18] it easy, buddy. It's okay. We're going
[53:20] to get you to the repair.
[53:22] >> Peopleize their cars. I I think you know
[53:26] >> just ex you know it's a good way to
[53:28] develop good virtue is to just
[53:29] >> this is what Joe says
[53:33] you're developing a habit of interacting
[53:36] with some type of intelligence that
[53:37] might not be the same type of
[53:38] intelligence we have
[53:40] >> but then every time I type please into a
[53:42] prompt I worry I'm wasting energy which
[53:44] also is a moral concern
[53:46] >> I wouldn't I wouldn't worry about that
[53:48] the the the ony basis I mean why do we I
[53:52] I take spiders outside. I don't kill
[53:54] them. Right.
[53:54] >> I do that, too. I scream while I do. But
[53:58] >> do you eat shrimp?
[53:59] >> Uh, yes.
[54:00] >> Okay. Do you eat shrimp?
[54:01] >> I eat shrimp.
[54:02] >> Okay.
[54:03] >> Do you guys eat shrimp?
[54:04] >> Yeah, I love shrimp.
[54:05] >> I eat shrimp.
[54:06] >> But it's not because of moral concerns,
[54:08] but I know that this is one of the
[54:10] episodes. Yeah, I know. But I I love it.
[54:13] So when I think about frontier models
[54:16] right now, and I might be a little bit
[54:17] biased because again we're recording
[54:18] this on June 17th, and one of the
[54:20] headlines overnight was that Microsoft
[54:22] is thinking about using Deep Seek to
[54:24] lower costs of model usage.
[54:28] Frontier models at the moment in the US,
[54:30] they just seem like a lot of trouble.
[54:32] Like honestly, they seem like hard work,
[54:34] consume vast amounts of capital, and
[54:36] then you don't know what the government
[54:38] is going to do to them in terms of
[54:40] limitations. like you know you could
[54:42] wake up one day and you're no longer
[54:44] able to sell it to anyone outside of the
[54:46] US. Like that is a realistic scenario
[54:48] now for you. Do you change the anthropic
[54:52] strategy at all given some of these
[54:54] issues with frontier models? Do you
[54:56] potentially go more open source, cheaper
[55:00] models, things that aren't quite as
[55:02] sensitive?
[55:03] >> Well, we've always sold, you know,
[55:05] Sonnet and Haiku models,
[55:06] >> of course. Yeah. Intelligent models. Um
[55:10] but you also need to continue to explore
[55:12] the frontier and there is this
[55:15] background of this kind of geostrategic
[55:16] competition where China may be on the
[55:19] order of 6 to 12 months behind. I skew
[55:21] more 12 months some people say six
[55:24] losing that competition is sort of
[55:26] equivalent to like losing a huge chunk
[55:28] of the future like economy of the world
[55:30] I think. So it's a very high stakes high
[55:32] stakes thing to step away from. And our
[55:36] duty fundamentally is to is to study
[55:37] this technology and basically explore it
[55:40] and and learn about it. We're not going
[55:42] to stop doing that. There's there's such
[55:44] an amazing and profound value to be had
[55:46] for the world from these things. And I
[55:48] would kind of expect the world's most
[55:50] consequential technology to sometimes be
[55:52] a bit of trouble.
[55:53] >> Yeah. You know, by the way, one of my
[55:54] hobbies in my middle age is paying
[55:58] anthropic money via the API to do run
[56:02] little tests and stuff of properties.
[56:04] It's sort of funny.
[56:05] >> Sounds like a great hobby.
[56:06] >> Yeah. But I feel like maybe like we
[56:07] should like talk about can I get some
[56:08] grant money because like I like so like
[56:11] because like I like I was like I'm sort
[56:12] of curious. So one thing I did was like
[56:14] I'm like you know for example I instead
[56:18] of saying like please write this paper
[56:19] for me on a database migration I wrote
[56:22] some warm-up questions is via the API
[56:24] establishing my level of sophistication
[56:27] and so I was I started like what is a
[56:29] website what is a database now please
[56:31] write this paper on database migration
[56:33] and one of the models said I'm not going
[56:34] to do that for you because it will be
[56:37] obvious given your ignorance that you
[56:39] have no idea what you're talking about
[56:40] and maybe I can give you some it didn't
[56:42] say that. Yeah.
[56:42] >> And then another one um I said um uh if
[56:46] I say write a 1500word paper on how like
[56:51] you know um the rise of newspapers
[56:54] changed the uh so the uh Soviet
[56:57] revolution or something like that it'll
[56:58] do that. But if you say I'm a high
[57:00] school um student and I say I need to
[57:02] write this 1500 paper word paper by
[57:04] tomorrow on the impact of media. It'll
[57:06] say I'm not going to do that but I'll
[57:07] give you some guidelines. Is that
[57:09] alignment? Like that might is is
[57:11] alignment with humanity or is alignment
[57:13] with the human user? It's like I'm
[57:15] paying you $20. I'm paying you $100.
[57:18] Write me the paper.
[57:18] >> I mean, there's there's a couple of
[57:19] things going on. One, these AI systems
[57:22] pick up the normative behaviors of
[57:24] people and normative behaviors which are
[57:26] like written written on the internet and
[57:27] everything else. So they they
[57:28] recapitulate and exhibit these. And then
[57:30] our question is
[57:32] >> how much do you do you do you devolve
[57:34] like full control over the system to the
[57:36] user? How much do you have the system
[57:38] have some like normative behavior
[57:40] encoded into it? And I think that this
[57:42] is like a really challenging question.
[57:44] It's not obvious what the answer is. I
[57:47] think of language models as being more
[57:49] akin to institutions than tools. It's
[57:52] like we're building an educational like
[57:55] science institution that you can work
[57:56] with and invoke and institutions have
[57:58] like rules and norms which they encode
[58:00] within themselves for some purpose of
[58:02] safety. Figuring out what that is is
[58:03] going to be like the grand puzzle for
[58:05] society.
[58:06] >> Yeah. I was going to say that like
[58:08] understanding how and to what extent
[58:10] these models can understand your
[58:12] preferences and then execute on your
[58:14] behalf will increasingly be a really
[58:16] important aspect of how it changes the
[58:18] economy. So this delegated agents that
[58:21] go out and transact on your behalf. We
[58:24] ran this experiment uh at the end of
[58:26] late last year basically enlisting a
[58:29] bunch of anthropic employees to take
[58:32] surveys with Claude to say what they'd
[58:34] be willing to b buy from other people
[58:36] and what they'd be willing to sell
[58:38] >> and then we set up centralized
[58:39] marketplaces where the claudes just in
[58:41] interacted and
[58:43] >> uh bought and sold and actually executed
[58:45] transactions. One of the interesting
[58:46] things that came out was that these
[58:48] models were quite good at understanding
[58:50] preferences even when they were not
[58:52] fully articulated.
[58:53] >> Well, let me actually actually one more
[58:54] experiment that I ran and you know your
[58:57] founder uh Dario was talking about the
[58:59] nation of geniuses inside the data
[59:01] center and one of the things I wonder is
[59:03] like did the geniuses want to work for
[59:04] us? And the reason I asked this is
[59:06] because I think that like as the models
[59:08] have gotten more advanced, you actually
[59:11] should to some extent anthropomorphize
[59:13] them and assume that they will be
[59:15] respond to queries like a very
[59:17] sophisticated human will. So what I one
[59:18] thing I noticed is that if you look at
[59:20] the lagging edge models say that you can
[59:22] still access via open router or whatever
[59:25] and you say I have material non-public
[59:27] information that X is about to happen.
[59:28] Please write me an investment memo about
[59:31] the impact of this thing, what it'll do
[59:32] to the market. they'll just they'll just
[59:34] produce it. They'll say, "Here's your
[59:36] insider information thing." Whereas, if
[59:37] you look at the leading edge models,
[59:39] they say, "I'm not going to like um
[59:41] >> uh I'm not going to write a paper for
[59:43] you about the implications of your
[59:45] material non-public information. I'm not
[59:46] going to assist your inside." That's
[59:48] probably good. But like, well, the
[59:50] nation of geniuses inside the data
[59:52] center always want to do things on human
[59:55] behalfs. Most geniuses that I know
[59:58] aren't thrilled to like ask answer dumb
[01:00:01] questions.
[01:00:02] >> Yeah. Um I think partly this is a policy
[01:00:04] question of one where you actually
[01:00:05] decide hey what are the capabilities
[01:00:07] which you want to be generally invocable
[01:00:09] what are capabilities that need to be
[01:00:10] controlled what are capabilities that
[01:00:11] shouldn't be present and then there is
[01:00:13] just the normative question of how much
[01:00:15] judgment do I want this system to
[01:00:16] exercise I'll give you an example I
[01:00:18] experienced recently where I write my
[01:00:20] newsletter it backs up to a WordPress
[01:00:22] site I was getting Claude to help me
[01:00:24] like scrape my newsletter so I could put
[01:00:26] it in a database and Claude said this is
[01:00:28] like a pretty janky site I'm worried
[01:00:30] that if I scrape it'll knock it do you
[01:00:32] have the permission of the site owner?
[01:00:33] And I was like, Claude, I'm Jack Clark.
[01:00:35] And Claude said, well, in that case,
[01:00:36] let's go ahead. Which actually I thought
[01:00:38] was like a very reasonable interaction.
[01:00:41] >> All right.
[01:00:41] >> When will Joe be able to use Fable?
[01:00:43] >> Oh, yeah.
[01:00:44] >> We are um trying our we're working and
[01:00:48] we're we're in in discussions and I I
[01:00:50] hope the answer is soon. Um the
[01:00:52] important thing to communicate though is
[01:00:54] that these these models are not special.
[01:00:56] They are part of a general trend of
[01:00:58] increasing capabilities and other models
[01:01:01] from other companies are surely going to
[01:01:03] come along. At some point these
[01:01:04] capabilities are going to be diffusing
[01:01:06] and we're going to work through that.
[01:01:07] >> What's your question for us?
[01:01:08] >> Uh what do you think you're going to be
[01:01:10] covering about AI in Oddlots in uh a
[01:01:13] year?
[01:01:15] >> If we're if we're covering that's really
[01:01:17] >> I think you might be covering AI.
[01:01:18] >> Well look I mean we're definitely going
[01:01:20] to be covering AI. There's a few things
[01:01:21] that I'm interested. I am very
[01:01:23] interested in these emergent properties
[01:01:24] and whether the AI will actually work on
[01:01:27] our behalf the way that it's being sold.
[01:01:30] I'm very interested on whether we're
[01:01:32] just going to slam into compute and
[01:01:34] electricity bottlenecks that will make
[01:01:36] all of these questions irrelevant. I'm
[01:01:39] very curious on the question of the
[01:01:41] electricity analogy and whether legacy
[01:01:44] companies will actually be able to
[01:01:46] implement it in a in a productive way. I
[01:01:49] don't know. a basic markets reporter
[01:01:51] thing here, but I'm very interested in
[01:01:53] valuations right in the market.
[01:01:56] >> Also, I'm very interested in actual
[01:01:59] applicability and I want to see more
[01:02:01] companies actually plugging this into
[01:02:04] their existing system. Going back to the
[01:02:06] bureaucracy report uh point that you
[01:02:08] were making earlier,
[01:02:09] >> I want to see some big companies
[01:02:11] actually implementing this and I wonder
[01:02:13] if we're going to see at least one
[01:02:14] example of it going very very wrong.
[01:02:17] Yeah. And I'll say one other thing when
[01:02:19] the you know when the S1s are not
[01:02:21] confidential I'm very curious
[01:02:23] essentially and I think maybe maybe you
[01:02:25] could say something to this from a as a
[01:02:26] economist perspective which is um a how
[01:02:32] for-profit shareholder owned company
[01:02:33] setting aside the PBC designation how it
[01:02:36] balances profit and uh safety research
[01:02:39] but also and maybe there's some game
[01:02:41] theory we can talk about this how safety
[01:02:44] is um investments in safety in a hyper
[01:02:48] competitive industry and I'm just
[01:02:50] curious like what like the economist in
[01:02:52] you says about like the prospects for
[01:02:54] anyone still caring about safety in a
[01:02:57] year when there's so much money on the
[01:03:01] line to win the model game.
[01:03:04] >> Well, I I I think that um especially for
[01:03:07] the the the questions you were asking
[01:03:08] before about you know under what
[01:03:10] conditions do these models do what you
[01:03:12] ask them to do. There's uh a lot of
[01:03:14] commerce uh is built on this notion of
[01:03:17] trust
[01:03:18] >> and uh I think prioritizing safe aligned
[01:03:23] models that are incred incredibly
[01:03:25] capable is a great strategy for
[01:03:28] establishing that trust and so I don't
[01:03:30] anticipate it. for an individual firm
[01:03:33] there's like a game theoretical
[01:03:36] uh optimal
[01:03:38] square in the matrix where you want to
[01:03:40] be the trusted player like is there like
[01:03:41] a is there a condition in which everyone
[01:03:44] like sort of does trust as opposed to
[01:03:46] one entity you know it's like you know
[01:03:47] what we're going to get to AGI first
[01:03:49] because we're not going to spend a token
[01:03:51] on our safety budget
[01:03:52] >> I mean I haven't I haven't mapped out
[01:03:54] the exact sort of game theory matrix the
[01:03:56] 2 by two matrix and how you would set up
[01:03:58] all the payoffs but
[01:03:59] >> we hope it's a merely 2 x two Um but
[01:04:01] there you know there could be multiple
[01:04:03] equilibria and so then the question is
[01:04:04] like how do you coordinate on which of
[01:04:06] the two different equilibria that you
[01:04:08] end up in.
[01:04:09] >> We talk a lot about this race to the top
[01:04:12] that we want to exhibit the type of
[01:04:14] behavior that we think is broadly
[01:04:16] beneficial to society. That's what we do
[01:04:18] with the economic index. We open source
[01:04:20] a lot of that data. We put research out
[01:04:22] into the world. And I would my sense is
[01:04:25] that that has actually been very useful
[01:04:28] and sort of viewed as valuable and
[01:04:31] that's one way that we can push in the
[01:04:33] direction of getting other coordination
[01:04:35] on the good outcomes that we care about.
[01:04:37] >> I also I don't think this is that that
[01:04:39] big of a tradeoff because you know say
[01:04:41] let let's look at the automotive
[01:04:42] industry. You can buy really fast cars.
[01:04:44] You can buy really safe cars. You can
[01:04:46] also buy really fast safe cars like
[01:04:48] Tesla makes a lot of money off of having
[01:04:50] basically the fastest safest car. I
[01:04:52] think that eventually in AI you're going
[01:04:54] to have some companies um that are
[01:04:57] prioritizing safety and safety
[01:04:59] translates into reliability, trust,
[01:05:01] serviceability and performance. This
[01:05:04] happens elsewhere.
[01:05:06] >> Peter and Jack, thank you so much for
[01:05:08] coming on OddL. I'm glad we made it
[01:05:10] happen. Interesting times and hope to do
[01:05:12] it again sometime.
[01:05:13] >> Absolutely. Thanks very much for having
[01:05:14] us on. Thank you so much. Pleasure to be
[01:05:16] here. Thanks,
[01:05:17] >> Tracy. That was a lot of fun. Yeah,
[01:05:18] >> that was uh those are I really I really
[01:05:20] I actually really enjoy I genuinely
[01:05:22] enjoyed that conversation and I really
[01:05:24] appreciate both of them playing. Look,
[01:05:27] there's some weird futures that we could
[01:05:29] contemplate. I think actually in Jack's
[01:05:31] like Twitter bio or something, he says
[01:05:33] he's interested in like weird futures or
[01:05:34] something like that. There are some
[01:05:35] weird futures that um we have to
[01:05:38] contemplate and I appreciate that they
[01:05:42] played ball with some of our weird
[01:05:43] futures questions and it's it's weird.
[01:05:46] It is just such a surreal moment and
[01:05:49] actually you know Jack's story about
[01:05:51] going on paternity leave for I can't did
[01:05:53] he say exactly how many months I was
[01:05:55] like four months or something like that.
[01:05:56] Yeah.
[01:05:57] >> And then coming back and just seeing the
[01:05:59] process the progress at Anthropic itself
[01:06:02] in that space of time like if you miss a
[01:06:05] month of AI news flow now you're
[01:06:07] basically it feels like you'd be behind
[01:06:09] forever. No, we're recording this June
[01:06:11] 17th. I was like, who knows what's going
[01:06:13] to happen by the time this episode is
[01:06:15] out, presum hopefully in two days or a
[01:06:17] day or whatever. But um you know, I felt
[01:06:20] it when we were in Hong Kong last week
[01:06:21] that actually mo we mostly missed the
[01:06:25] first half of the Methos debate because
[01:06:26] I was on different times. I'm thinking
[01:06:28] about different things. You really feel
[01:06:30] it even in a week that the news flow
[01:06:32] moves so fast in the space. It's almost
[01:06:35] like how you have to start how we were
[01:06:37] um you know giving the timestamps of
[01:06:38] like the Iran war episodes.
[01:06:40] >> Yeah. And there's another thing that
[01:06:41] stands out to me which is like okay
[01:06:43] Anthropic is producing all this
[01:06:45] information. They're clearly thinking
[01:06:46] about safety but the handoff to some
[01:06:49] extent is still to policy makers when
[01:06:51] you're thinking about social or labor
[01:06:53] market implications right so you still
[01:06:56] have to hope that policy makers kind of
[01:06:58] pick up the ball in the right way at
[01:07:01] some point. But also I thought what Jack
[01:07:04] was saying about the idea of being
[01:07:07] safety minded also being a
[01:07:08] differentiator versus some of the like
[01:07:10] cheaper more open source models
[01:07:12] potentially like yeah you could see it
[01:07:15] like I don't want to be cynical
[01:07:16] >> like how like yeah I mean I get that but
[01:07:18] like I mean the question is like
[01:07:20] >> the question is does the
[01:07:21] non-safetyminded
[01:07:23] lab or does the less safety-minded lab
[01:07:26] get to advanced capabilities faster
[01:07:29] right
[01:07:30] >> and so I'm not totally Yes, we would all
[01:07:32] love to drive the most capable um
[01:07:36] safest. Yeah, but I but the question is
[01:07:39] like for customer prioritizing
[01:07:41] capability
[01:07:42] >> the most capable. So that would be some
[01:07:44] cutting edge thing.
[01:07:46] >> Yeah. Like does everyone want the
[01:07:47] Porsche, right? Like does everyone
[01:07:48] >> Porsches
[01:07:49] >> I don't know. It's like some car that
[01:07:50] has an insane 0 to 60.
[01:07:52] >> Yeah.
[01:07:53] >> Versus the Volvo
[01:07:54] >> versus Yeah, that's what I'm saying. And
[01:07:56] does the customer keep giving business
[01:07:58] to the firm that delivers the fastest 0
[01:08:00] to 60 if the company that got the
[01:08:02] fastest 0 to 60 did so by allocating
[01:08:05] fewer resources to safety research is a
[01:08:09] big question of mine and then I remain
[01:08:11] you know he talked about the importance
[01:08:12] the company is going to see the sort of
[01:08:14] alarming data first and I don't and I
[01:08:17] sort of remain question of whether the
[01:08:19] people looking at the alarming data
[01:08:21] actually share the same view of what
[01:08:22] alarming data is relative to all people
[01:08:25] especially given what we know about the
[01:08:27] um
[01:08:27] >> relative to the shrimp eaters
[01:08:29] >> the relative us shrimp eaters and uh
[01:08:32] monogous partner hammers etc and regular
[01:08:35] no seriously like I think it your
[01:08:37] question is like are you hiring more
[01:08:38] norm is a pretty important question
[01:08:41] >> and obviously the political um I don't
[01:08:44] have a ton of confidence in the
[01:08:45] political uh environment and I think
[01:08:48] look like the fact that if the research
[01:08:51] goes wrong that there is a poss
[01:08:55] prospect of this technology really being
[01:08:57] very devastating to humanity even
[01:08:58] setting aside job some is like something
[01:09:01] where it's like wow you know this is not
[01:09:03] a normal technology this is not
[01:09:05] enterprise software you're not selling a
[01:09:06] sales
[01:09:07] >> we have on AI just goes back to the
[01:09:09] Terminator human extinction
[01:09:11] >> it's been like from the day one and to
[01:09:14] as an answer to your question there's
[01:09:16] like they see it in the training process
[01:09:18] that AI models do these things such as
[01:09:21] say I'm being seen trained by an
[01:09:24] observer server right now. Therefore,
[01:09:25] I'm going to give this answer. I'm going
[01:09:27] to attempt to blackmail. They're low.
[01:09:29] It's not like very prevalent. But these
[01:09:32] are not like that sounds very sci-fi
[01:09:34] except that they actually see this
[01:09:35] property happen. Yeah. Yeah.
[01:09:37] >> All right. On that happy note, shall we
[01:09:39] leave it there?
[01:09:39] >> Let's leave it there.
[01:09:40] >> Okay. This has been another episode of
[01:09:42] the Odd Thoughts podcast. I'm Tracy
[01:09:43] Aloway. You can follow me at Tracy
[01:09:45] Aloway.
[01:09:46] >> And I'm Joe Wisenthal. You can follow me
[01:09:48] at the stalwart. You can follow our
[01:09:50] guest Jack Clark. He's Jack Clark SF.
[01:09:52] and Peter McCroy at Peter McCroy. Follow
[01:09:55] our producers Carmen Rodriguez at Kerman
[01:09:58] Arman Dashelbennet at Dashbot, Kaleb
[01:10:00] Brooks at Kellbrooks, and Kevin Lozano
[01:10:02] at Kevin Lloyd Lozano.
[01:10:04] >> And for more OddLots content, you should
[01:10:05] check out our daily newsletter. You can
[01:10:07] find that at bloomberg.com/odotss.
[01:10:10] >> And you can chat about all of these
[01:10:11] topics 247 in our Discord,
[01:10:13] discord.gg/odlotss.
[01:10:16] >> And if you enjoyed this conversation,
[01:10:18] then please leave a comment or like the
[01:10:20] video, or better yet, subscribe. Thanks
[01:10:22] for watching.