Anthropic CEO Dario Amodei: AI's Potential, OpenAI Rivalry, GenAI Business, Doomerism

Channel: Alex Kantrowitz

Published at: 2025-07-30

YouTube video id: mYDSSRS-B5U

Source: https://www.youtube.com/watch?v=mYDSSRS-B5U

I get very angry when people call me a
doomer. When, you know, when when
someone's like, "This guy's a doomer. He
wants to slow things down." You you
heard what I just said, like, you know,
my my father died because of, you know,
cures that, you know, could have could
have happened a few years later. I
understand the benefit of this
technology. I'm sure you've heard the
criticism from people like Jensen who
say, "Well, Daario thinks he's the only
one who can build this safely and
therefore wants to control the entire
industry."
>> I've never said I've never said anything
like that. That's an outrageous lie.
That's the most outrageous lie I've ever
heard.
>> Anthropic CEO Dario Ammoday joins us to
talk about the path forward for
artificial intelligence. Whether
generative AI is a good business and to
fire back at those who call him a
doomer. And he's here with us in studio
at Anthropic headquarters in San
Francisco. Dario, it's great to see you
again. Welcome to the show.
>> Thank you for having me.
>> So, let's recap the past couple months
for you. Uh, you said AI could wipe out
half of entry-level white collar jobs.
You cut off Windsurf's access to
Anthropic's top tier models when you
learned that OpenAI was going to acquire
them. You asked the government for
export controls and annoyed Nvidia CEO
Jensen Wong. Uh, what's gotten into you?
>> Um, you know, I think I think Anthropic,
myself and Anthropic are always focused
on kind of trying to do and say the
things that we believe. Um uh and I
think as we've gotten uh more close to
AI systems that are more powerful um you
know I think I've wanted to say those
things um you know more uh more
forcefully more publicly to make the
point clearer. Um you know I've been
saying for many years that you know we
have these we can talk in detail about
them but you know we have these scaling
laws. AI systems are getting more
powerful. They're going from the level
of, you know, a a few years ago they
were barely coherent now, you know, a
couple years ago they were at the level
of a smart high school student. Now
we're getting to smart college student,
PhD, and they're starting to they're
starting to apply across the economy.
And so I think all the issues related to
AI ranging from kind of the national
security issues to the economic issues
um you know are are are starting to
become quite near to where to where we
um you know to where we're actually
going to face them. And so and so I
think as these problems have come closer
I've you know even though you know in
some form anthropic has been saying
these things for a while I think the
urgency of these things has gone up and
and and and and you know I I I want to
make sure that we uh you know I want to
make sure that we say what we believe
and that we warn the world about the
possible downsides even though you know
no one can say what's going to happen.
we're, you know, we're saying what we,
you know, what we think might happen,
what we think is likely to happen. You
know, we we back it up as as best we
can, although it's often, you know,
extrapolations about the future where
where no one where no one can be sure.
Um, but, you know, I think we we see
ourselves as kind of as kind of having a
duty to, you know, to kind of warn the
world about what's going to happen. And
that's not to say, you know, I think
there's an incredible number of like
positive applications of AI, right? I've
I've kind of continued to talk about
that. I read this I wrote this essay,
Machines of Loving Grace. Um I I I feel
in fact that I and anthropic have often
been able to do a better job of
articulating the benefits of AI than
some of the people who call themselves
optimists or accelerationists. Um so I
think we probably appreciate the
benefits more than more than anyone. Um,
but for exactly the same reason, because
we can have such a good world if we get
everything right, I feel obligated to
warn about the risks.
>> So all of this is coming from your
timeline. Basically, you it seems like
you have a shorter timeline than most
and so you were feeling a sense of
urgency to get out there because you
think that this is imminent.
>> Yes, I'm not sure. Um, you know, I think
it's very hard to predict particularly
on the societal side. So if you say you
know when are people going to deploy AI
or when are companies going to use you
know XX X dollars of spend of AI or you
know when will when will AI um you know
be be be used in these applications or
when will it drive these medical cures
that's kind of harder harder to say I
think the underlying technology is more
predictable but still uncertain still no
one knows but I think on the underlying
technology I've started to become more
confident. There isn't no uncertainty
about it. You know, I think the the
exponential that we're on could could
kind of still um you know, could totally
peter out. You know, I think there's
maybe uh I don't know 20 or 25% chance
that sometime in the next 2 years the
models just start getting stop getting
better for reasons we don't understand
or maybe reasons we do understand like
you know data or compute availability
and then everything I'm saying just just
seems seems totally silly and everyone
makes fun of me for all the warnings
I've made and and you know I'm just I'm
just totally fine with that given given
the distribution that that that given
the distribution that I see
>> and so I should say that this is part
our conversation as part of a profile
I'm writing about you. I've spoken with
more than two dozen people who've worked
with you, who know you, who've competed
with you, and I'm going to link that in
the show notes if anybody wants to read
it. It's free to read. Uh, but one of
the themes that has uh come through
across everybody I've spoken with is
that you have about the shortest
timeline of any of the major lab leaders
and you just referenced it uh just just
now. So, why do you have such a short
timeline and why should we believe in
yours? Yeah, it it really depends what
you mean by timeline. Um, so one thing
and you know I've I've I've been
consistent on this over the years is you
know there are these terms in the AI
world like AGI and super intelligence.
Like you'll hear leaders of companies
say we've achieved AGI, we're moving on
to super intelligence or like it's
really exciting that someone stopped
working on AGI and started working on
super intelligence. So I think these
terms are totally meaningless. I don't
know what AGI is. I don't know what
super intelligence is. It it sounds like
a it sounds like a marketing term. Yeah,
it sounds like, you know, something
something designed to activate people's
people's dopamine. So, you'll see in
public I never use those terms and uh I
um you know, I'm I'm actually, you know,
careful to criticize the use of those
terms. Um but I think I think despite
that, I am I am indeed one of the most
bullish about about AI capabilities
improving very fast. The thing I think
is real that I've said over and over
again is the exponential. The idea that
every few months we get an AI model that
is better than the AI model we got
before. And that we we get that by
investing more compute in AI models,
more data, more new types of training
models. Initially, this was done by
what's called pre-training, which is
when you just feed a bunch of data from
the internet into the model. Now we have
a second stage that's reinforcement
learning or test time compute or
reasoning or whatever you want to call
it. I I think of it as a second stage
that involves reinforcement learning.
Now both of those things are scaling up
together as we've seen with our models
and as we've seen with models from other
from other companies and I don't see
anything blocking that the further
scaling of that. There's some stuff
about you know how do we broaden the
tasks on the RL side of it. we've seen
uh more progress on say math and code
where where the models are you know
getting pretty close to like a high
professional level and less on more
subjective tasks but I think that is
very much a temporary obstacle um uh so
when I look at it I see this exponential
and I say look people aren't very good
at making sense of exponentials right
like you know if something is doubling
every six months then uh you know two
years before it happens it looks like
it's only 1/16th of the way there and
and and so we are sitting here in the
middle of of 2025
um and the models are really starting to
explode in in in terms of the economy
right if you look at the capabilities of
the model they're starting to saturate
all the benchmarks if you look at
revenue and you know anthropics revenue
every year has grown 10x um uh every
year we're we're kind of conservative
and we say you know it can't grow 10x
this time you know I I I you know I
never I never assume anything and and
actually always am very conservative in
saying ah I think it's going to slow
down on the business side but we went
from zero to 100 million in 2023 we went
from 100 million to a billion in 2024
and you know this year in this first
half of the year we've gone from 1
billion to you know I think as of as of
speaking today it's it's well above four
it might be 4.5 um uh and so if you
think about it you know suppose that
exponential continued for two years I'm
not saying it will but suppose it
continued for two years you know you're
you're like well into the hundred
billions. I'm not saying that'll happen.
I'm saying the situation is that when
you're on an exponential, you can really
get fooled by it. Two years away from
when the exponential goes totally crazy.
It it, you know, it looks like it's just
starting to be a thing. Um, and so
that's the fundamental dynamic. You
know, we saw that with the internet in
we saw that with the internet in the
'9s, right? where it was like you know
networking speeds and the underlying
speed of the computers were getting fast
and over a few years it became possible
to have to basically build a digital
global communications network on top of
all this when it wasn't possible just a
few years ago and and almost no one
except for a few people really saw the
implications of that and how fast it
would happen and so that's that's where
I'm coming from that that's what I think
now I don't know like if a bunch of
satellites crashed maybe the internet
would have taken longer. If there was an
economic crash, maybe it would have
taken a little longer. So, we can't be
sure of the exact timelines, but I think
people are getting fooled by the
exponential and and not realizing how
fast it it might be. How fast I think it
probably will, although I'm not sure.
>> But so many folks in the AI industry are
talking about diminishing returns from
scaling. Now, that really doesn't fit
with the vision you just laid out. Are
they wrong?
>> Yeah. uh I I from what we've seen I can
only speak in terms of the models at
enthropic um but what I think seen in
terms of the models at enthropic if we
look at you know let's take coding
coding is one area where you know I
think anthropic models have advanced
very quickly adoption has been very
quick we're not just a coding company
we're planning to expand to many areas
but if you if you look at if you look at
coding um you know every you know we
release 3.5 sonnet a model we call 3.5
sonnet V2 um uh which I you know let's
call it 3.6 Sonnet now um 3.7 sonnet uh
and then 4.0 sonnet and 4.0 obus and you
know that series of four or five models
each one got substantially better at
coding than than the last. If you want
to look at benchmarks, you can look at,
you know, SweetBench growing from uh,
you know, I think 18 months ago was at
like 3% or something um, growing all the
way to, you know, 72 to 80% depending on
how you how you measure it and and the
real usage has grown grown exponentially
as well where we're heading more and
more towards autonomously you can just
use these models. I think the actual
majority of code um uh uh that's written
written at Enthropic uh is you know at
this at this point uh written by or at
least with the involvement of one you
know one of the clawed models um and
various uh various other companies have
have said you know have said you know
similar similar statements to uh similar
statements to that. So we see the
progress as being very fast and the
exponential is continuing and and we
don't see any diminishing returns.
>> But there are some liabilities it seems
like with large language models. For
instance, continual learning. We had
Dark Kesh on a couple weeks ago. Here's
how he put it and he wrote about it in
his Substack. The lack of continual
learning is a huge huge problems. The LM
baseline at many tasks might be higher
than an average human, but you're stuck
with the abilities you get out of the
box. So you just make the model and
that's it. It doesn't learn. that seems
like a glaring liability. What do you
think about that?
>> So, first of all, I would say even if we
never solved continual learning, um even
if we never solve continual learning and
memory, um I think that the potential
for the LLMs to do, you know, incredibly
uh incredibly well to, you know, affect
things at the scale of the economy will
be very high. Right? If I think of the
field I used to be in biology and
medicine like you know let's say I had a
very smart Nobel prize winner and you
know I I I said okay you know you're you
know you've you you've discovered all
these things you have this incredibly
smart mind but you know you can't you
can't you know you can't like read new
textbooks or absorb any new information.
I mean that would be difficult but like
still if you had like 10 million of
those like they're they're still going
to make a lot of biology breakthroughs.
Like they're going to be limited.
they're going to be able to do some
things humans can't and there are some
things humans can do that they can't.
But but but even that even if we impose
that as a ceiling like man that's pretty
damned impressive and transformative and
even if I said you never solve that like
I think you know I think people are
underestimating the impact but but look
context windows are getting longer and
models actually do learn during the
context window right so so as as I you
know talk to the model during the
context window I have a conversation it
absorbs information the underlying
weights of the model may not change but
but you know just just like I'm talking
to you here and we're having a
conversation and I listen to the things
you say and I I you know I think and and
I like respond to them. The models are
able to do that and and from a from an
from a machine learning perspective,
from an AI perspective, there's no
reason we can't make the context length
a 100red million words today, right?
Which is roughly what a human hears what
what a human hears in their lifetime. Um
uh there's no reason that we can't uh do
that. It's it's really inference
support. Um uh and so again even that
fills in many of the gaps not all the
gaps but it fills in many of the gaps
and then there are a number of things
like learning and memory that do allow
us to update the weights. So um you know
there there there are a number of things
around you know types of reinforcement
learning learning training you know you
know we used to many years ago talk
about inner loops and outer loops right
the inner loop is like I have some
episode and I learn some things in that
episode and I'm trying to optimize for
the lifetime of that episode and and
kind of the outer loop is is is the
agents learning over episodes. Um and so
I think maybe that inner loop outer loop
structure is a way to learn the
continual learning. One thing we learned
in AI is whenever it feels like there's
some fundamental obstacle like two years
ago we thought there was this
fundamental obstacle around uh around
reasoning turned out just to be just to
be RL you just train with RL and you let
the model write some stuff down you know
you you let the model write things down
to try and figure out objective math
problems um without being too specific
um you know I think and we already have
maybe some some you know some some
evidence to suggest that this is another
of those problems that is is not as
difficult as it seems that will fall to
scale plus a slightly different way of
thinking about things. Do you think your
obsession with scale might blind you to
some of the new techniques like Deis
Sabis says, you know, to get to AGI or
you might call it super powerful AGI,
whatever human level intelligence is
what we're all talking about. We might
need a couple new techniques for that to
happen. If you're
>> developing new techniques, we're
developing new techniques every day.
>> Okay.
>> Um uh you know, Claude is very good at
code and we you know, we don't really
talk externally that much about why why
Claude is so good at code.
>> Why is it so good at code? Um it
>> like I said we don't talk externally
about it.
>> Um uh uh uh so you know every new
version of claude that we make has you
know improvements to the architecture,
improvements to the data that we put
into it, improvements to the methods
that we use to train it. So we're
developing new techniques all the time.
Um new techniques are a part of every
you know model that we build. And you
know that's why we you know I've said
these things about like you know we're
trying to optimize for like talent
density as much as possible like you
need that talent density in order in
order to invent the new techniques. You
know, there's one thing that's been
hanging over this conversation, uh,
which is that maybe Anthropic is the
company with the right idea, uh, but the
wrong resources. Because you look at
what's happening with XAI and, um, and
inside Meta where, uh, Elon's built his
massive cluster, Mark Zuckerberg is
building this 5 gawatt data center and
they are putting so much resources
towards scaling up. Um, is it possible?
I mean anthropic obviously you have
raised billions of dollars but these are
trillion dollar companies.
>> Yeah. So we've raised I think at this
point a little short of $20 billion.
It's not bad.
>> So that's that's not that's not nothing.
And I would also say if you look at the
size of the data centers that we're
building with for example Amazon. Um I I
don't think our data center scaling is
substantially smaller than that of any
of the other companies in the space. Um
you know in many cases these things are
limited by energy. they're limited by
capitalization. Um, you know, when when
when when when people talk about, you
know, these large amounts of money,
they're they're talking about it over
over over several years, right? And when
you hear some of some of these
announcements, sometimes they're not
funded yet. They're, you know, we've
we've seen the size of the of the of the
data centers that folks are building and
we're we're actually we're actually
pretty confident that, you know, we're
we will be within we will be within a
rough range of the size of data centers
they build. You talked about talent
density. What do you think about what
Mark Zuckerberg is doing on the talent
density front? I mean, combining that
with these massive data centers, it
seems like he's going to be able to
compete.
>> Yeah. So, uh this is this is actually
very interesting because um you know,
one thing we noticed is that relative to
other uh companies um you know, I think
I think I think uh very you know, a lot
fewer people from Enthropic uh have been
have been caught by these. And it's not
for lack of trying. I've talked to
plenty of people uh you know who who got
these offers at Enthropic and and who
just turned them down. um who wouldn't
even talk to Mark Zuckerberg who said um
you know uh uh no I'm I'm I'm staying at
Anthropic and and our our general
response to this was you know I posted
something to the to the whole company
Slack where where where I said look um
you know we are not willing to
compromise our you know our compensation
principles our principles of fairness to
respond individually to these offers.
The way things work at Enthropic is
there's a series of levels. One
candidate comes in, they get assigned a
level, and we don't negotiate that
level. Um, uh, uh, because because we
think it's unfair. We want to have a
systematic way. If, you know, if Mark
Zuckerberg, you know, throws a dart at a
dart board and hits your name, that
doesn't mean that you should be paid 10
times more than the guy next to you
who's who's, you know, who's who's just
as skilled, who's just as talented. Um
uh and and and and my view of the
situation is that you know the only way
you can really be hurt by this is if you
allow it to destroy the culture of your
company by panicking by treating people
unfairly in an attempt to to defend the
company. Um and I think actually this
was a unifying moment for the company
where um you know we we we didn't give
in. we refused to compromise our
principles because we had the confidence
that people are enthropic because they
truly believe in the mission. Um uh and
and you know I think um I I that that
gets to kind of how I see this. I think
that what they are doing is trying to
buy something that cannot be bought. Um
and and that is alignment with the
mission. Um uh and you know I I you know
I you know I think there are selection
effects here like I you know I you know
are they getting the people who are most
enthusiastic who are most missional
aligned who are most excited to
>> but they have talent and GPUs you're not
underestimating them.
>> I I
we'll see how it we'll see how it plays
out. Um I am pretty bearish on what
they're trying to do.
>> So let's talk a little bit about your
business because a lot of people have
been wondering is the business of
generative AI a real thing? And I'm also
curious. I have questions all the time.
You talked about how much money you've
raised, close to 20 billion. Um, you've
raised three billion, three billion from
Google, 8 billion from Amazon, three and
a half billion uh from a new round led
by Lightseed who I've spoken with. Um,
what what is your pitch? Because you are
not part of like a big tech company.
You're out there on your own. Do you
just bring the scaling laws and say,
"Can I have some money?" So my my view
of this has my view of this has always
been um that uh talent is the most
important thing. Um so you know if you
go back three years ago um you know we
were in a position where we had raised
mere hundreds of hundreds of millions.
Open AAI had already raised, you know,
13 billion from from Microsoft. And of
course, you know, the the large hyper
cap tech companies were sitting on 100
billion, $200 billion. And and basically
the pitch we made then is we know how to
make these models better than others do,
right? There may be a curve. There may
be a curve of scaling laws. But but
look, if we are in a position where we
can do for a hundred million what others
can do for a billion and we could do for
10 billion what they can do for a
hundred billion then it's 10 times more
capital efficient to invest in entropic
than it is to invest in in these other
companies. Would you rather be in a
position where you can do anything for
10 times cheaper or where you start with
a large pile of money? If you can do
things 10 times cheaper, the the you
know the the the the the money is a
temporary defect that that that that you
can remedy. If you have this intrinsic
ability to build things for the same
price, much better than anyone else or
as good as anyone else for much lower
price. You know, investors aren't aren't
idiots or at least they aren't always
idiots. Um
>> depends which one you go to.
>> I'm not going to name any names. um uh
uh uh but um uh uh you know they they
basically understand the concept of
capital efficiency. Um and so we've been
in a position you know 3 years ago where
you know these differences were like a
thousandx and now you're saying you know
with $20 billion can you compete can you
compete with hundred billion um and and
and my answer is basically yes because
of the talent density. You know, I've
said this before, but uh you know, the
the the Anthropic is actually the
fastest growing software company in
history at the scale that it's at. Um so
we grew from zero to 100 million in
2023, 100 million to billion in 2024.
And this year we've we've grown from 1
billion to I think I said this before
4.5. So that that 10x a year I mean you
know every every you know every year I
like suspect that we'll grow at that
scale and and every year I'm almost
afraid to say it publicly because I'm
like no it couldn't it couldn't possibly
happen again. So you know I think I
think the growth at that scale like kind
of speaks for itself in terms of our
ability to compete with the big players.
>> Okay. So CNBC says 60 to 75% of
anthropic sales are through the API.
That was according to internal
documents. Is that still accurate? Um I
won't give exact numbers but the
majority does come through the API
although we also do have a flourishing
apps business and you know I think more
recently um the you know max tier which
power users use as well as claude code
which which coders use. So you know I
think we have a we have a thriving and
fast growing apps business but yes the
majority comes through the API. So
you're making the most pure bet on this
technology like you know OpenAI might be
betting on chat GPT and Google might be
betting on the fact that no matter where
the technology goes it can you know
integrate into Gmail and calendar. So
why have you made this bet on this the
pure bet on the tech itself?
>> Yeah I mean I would say I wouldn't quite
put it that way. I think we've I would
describe it more as we've bet on
business use cases of the model um more
so than we've bet on the API per se. And
it's just that the first business use
cases of the model come through the API.
So you know as you mentioned OpenAI is
very focused on the consumer side.
Google is very focused on kind of the
existing products that Google has. Our
view is that if anything the enterprise
use of AI is going to be greater even
than the consumer use of AI or I should
say the business use because it's
enterprise, it's startups, it's
developers and it's kind of you know
power users using the model model for
productivity. Um I I also think that uh
being a company that's focused on the
business use cases actually gives us
better incentives to make the models
better. Um a a thought a thought
experiment that I think is worth running
is you know suppose I have this model
and it's uh it's um you know it's it's
as good as an undergrad at biochemistry.
Um and then I improve it and it's as
good as a PhD student at at
biochemistry. If I go to a consumer,
right, if I give them the chatbot and I
say, "Great news. I've improved the
model from, you know, undergrad to
graduate level in in biochemistry, um,
you know, maybe I don't know 1% of
consumers care about that at all, right?
99% are just going to be like, I don't
understand it either way." Um, but now
suppose I go to Fizer and and I say, you
know, I've improved this from
undergraduate biochemistry to to
graduate biochemistry. Like, this is
going to be the biggest deal in the
world, right? They might pay 10 times
more for something like that. it might
have 10 times more value to them. And so
the general aim of making the models
solve the problems of the world to make
them smarter and smarter but also able
to to bring many of the positive
applications right the things I wrote
about in like machines of loving grace
of like solving the problems of biio
medicine solving the problems of
geopolitics solving the problems of you
know of economic development um you know
as well as more prosaic things like
finance or legal or productivity or
insurance. Um I think it gives a better
incentive to kind of develop the models
um as as far as uh as as as as far as
possible and I think in many ways it's
like it may even be a more a more a more
positive business. So I would say we're
making a bet on the business use of AI
because it's most aligned with kind of
the exponential.
>> Okay. Then briefly, how did you decide
to go with the coding use case?
>> Yes. So um you know uh originally as
happens with most things we're trying to
optimize for making the model better at
a bunch of stuff and you know coding
particularly stood out in in terms of
how how valuable it was. You know I've
worked with thousands of engineers and
there was a point about a year year and
a half ago where one of the best I'd
ever worked with said um uh you know
every previous coding model has been
useless to me and and this one finally
was able to do something I wasn't able
to do. And then after we released it, it
started getting quick adoption. This was
around the time that you know a lot of
the coding companies like Cursor,
Windsurf, GitHub, Augment Code started
started exploding in in popularity and
then when we saw how popular it was, we
kind of doubled doubled down on it. Um
my view is that coding is particularly
interesting because a the adoption is
fast um and and b getting better at
coding with the models actually helps
you to develop the next model. Um, so it
has it has a number of uh you know you
know I would say advantages
>> and now you're selling uh your AI coding
through claude code. Um but it's very
interesting the pricing model um has
been confounding to some. You can spend
$200 a month and get the equivalent I
spoke to one developer they got the
equivalent of $6,000 a month uh from
your API. Um, Ed Zitetron has pointed
out the more popular that your models
get, the more money you're going to lose
if people are super users of this
technology. So, how does that make
sense? So, um, uh, uh, so actually
pricing schemes and rate limits are
surprisingly complicated. Um so so some
of this is basically the result of when
we released our um uh when we when we
released claude code and the max tier
which we eventually tied together
actually not fully understanding the
implications of you know the ways in
which people could use the models and
how much they were actually able to get.
So over the last few days, as of the
time of this uh as of the time of this
interview, we've adjusted that
particularly on the larger models like
Opus, I think it's no longer possible to
spend that much um uh uh with a with a
with a $200 uh uh subscription. And you
know, it's possible more changes will
more changes will come in the future,
but we're always going to have a
distribution of users who use a lot and
and and users who lose who use some
amount. And it it doesn't necessarily
mean we're losing money that there are
some some users who get more um uh you
know who if you were to measure via API
credits spend you know get get a better
deal on the consumer on on the consumer
subscription than they would on the API
products. Right? There's there's a lot
of assumptions there. Um and I can tell
you that that some that some of them
that some of them are wrong. Um uh we
are not in fact uh losing money. But I
guess there's another question about
whether you can continue to serve these
use cases um and not raise prices. So uh
just to give you a couple stats, there
are some developers that are upset
because uh using Anthropic's newer
models in cursor is costing them more
than it ever has. U startups that I've
spoken with say Anthropic is down a much
down a bunch because they can't get
access to the GPUs. At least that's what
they um what they imagine is happening.
And I was just with Amjad Masad at
Replet um in an interview that we're
going to air next week who said there
was a period of time where the price per
token price per to use these models was
coming down and it stopped coming down.
So is it the is what's happening that
these models are just so expensive for
anthropic to run that it's hitting it's
a wall of its own
>> again I think you're I think you're
making assumptions here.
>> That's why I'm asking the CEO. Uh yeah
um uh you know uh you know the the the
way I think about it is we think about
the models in terms of how much value
are they creating right um so as the
models get better and better I think
about how much value they create and
there's a separate question about how
the value is distributed between those
who make the model those who make the
chips and and you know those who make
those who make the the the um the the
the the underlying applications. So
again, without being too specific, like
I think there are some assumptions in
your question that are not necessarily
correct. Um uh you know, I
>> tell me which ones
>> I I so I'll say this. I do expect the um
I I I expect the price of providing a
given level of intelligence to go down.
Um I expect the price of providing the
frontier of intelligence which will
which will provide kind of increasing
economic value that might go up or it
might go down. My guess is it probably
stays about where it is. But again the
value that's created you know goes way
up. So two two years from now my guess
is that we'll have models that cost of
the same order of magnitude that they
cost today except they'll be much more
capable of doing work much more
autonomously much more broadly than they
are capable of today. One of the things
that Amjad mentioned was he thinks that
the bigger models are not as intensive
to run or more intensive to run given
their size because of the architecture
and some of these techniques that we
talked about that they're lighting up
only certain sections of the model. So
his idea I'm hopefully conveying this
truthfully is that um Anthropic can run
these models without too much bulk on
the back end but is still keeping those
prices uh where they are. And I think
the the line that I'm going to draw
there is maybe that um to get to
software margins, there were some
reports that Anthropic is slightly below
software gross margins. You're going to
have to charge a little bit more for
these models.
>> So, um yeah, again, um I think larger
models cost more to run than smaller
models. Um uh uh you know, I think the
technique you're referring to is maybe
mixture of experts or something like
that. So whether your models are mixture
of experts or not, like mixture of
experts is is like a way to run models
more cheaply that have a given number of
parameters. It's a way to train models.
But if you're not using that technique,
then larger models that don't use that
technique cost more to run than smaller
models that don't use that technique.
And if you're using that technique,
larger models that use that technique
cost, you know, more to run than than
smaller models that that that are using
that technique. So I think I think
that's sort of a distortion of the I
think that's sort of a distortion of the
situation. Um
>> basically I'm I'm just guessing and I'm
trying to find out what the truth is
from you. So
>> yeah look so I um you know in in in in
terms of like the cost of the models
like one thing you'd be surprised by
people you know people kind of impute
this thing to like oh man it's going to
be really hard to get the margins from
like x% to y%. We make improvements all
the time that make the models like 50%
more efficient than they are before. We
are just the beginning of optimizing
inference. Um uh inference has improved
a huge amount where from from where it
was a couple a couple years ago to where
it is now. That's why the prices are
coming down.
>> And then how long is it going to take to
be profitable? Because I think the loss
is going to be like three billion this
year. That's what they would distinguish
different things. Okay.
>> Um there's the cost of running there's
the cost of running the model, right? So
so for every dollar the model makes it
costs a certain amount. Um that is
actually already fairly profitable. Um
there are separate things. There's you
know the cost of paying people um and
like buildings that is actually not that
that large in the scheme of things. The
big cost is the cost of training the
next model. Um, and I think this idea of
like the companies losing money and not
being profitable is it's a little bit
misleading. Um, and you start to
understand it better when you when you
look at the scaling laws. So, as a
thought exercise, these numbers are not
exact or even close for entropic. Let's
imagine that in 2023 you train a model
that costs $und00 million. Um, and then
in 2024, you deploy the 2023 model and
it makes $200 million in revenue, but
you spend a billion dollars to train um,
you know, to train a new model in 2024.
So, and then, you know, and then in
2025, the billion dollar model makes two
billion in revenue and you spend 10
billion to train the next model. Um, so
the company every year is unprofitable.
It lost 800 million in 2024 and then 20
2025 it lost$8 billion. Um so you know
this looks like a hugely unprofitable
enterprise but if instead I think in
terms of is each model profitable right
think of each model as a venture. Um I
invested 100 million in the model and
then I got then I got then I got 200
million out of the model the next year.
So that model had 50% margins and and
you know and like and like made me
hundred $100 million the next year. um
you know the the the company invested a
billion dollars and made and made $2
billion in in or sorry the next model
the company invested a billion dollar so
every model is profitable but the
company is unprofitable every year I'm
not I'm not this is this is this is a a
styliz I'm not like claiming these
numbers for anthropic or claiming these
facts for but this general philanthropic
this general dynamic is is this this
general dynamic is in general terms the
explanation for what is going on. And so
at you know you know at at any time if
the models stopped getting better or if
a company stopped investing in the next
model um you know you would you know you
would have probably a viable business
with the existing models but everyone is
investing in the next model and so
eventually it'll get it'll get to some
scale. the but the fact that we're
spending more to to to to this fact that
we're spending more to invest in in the
next model suggests that the the scale
of the business is going to be larger
the next year than it was than it was
the year before. Now, of course, what
could happen is like the models stop
getting better and there's this kind of
one-time cost that's like a boondoggle
and we spend we spend a bunch of money,
but then the the you know the the
companies the industry will kind of
return to this you know to this plateau
to this level of profitability or the
exponential can keep going. Um, so I
think I think that's a long-winded way
to say I don't think it's really the
right way to think about things,
>> right? But what about open source?
Because if you stopped, let's say you
stopped investing in the models and open
source caught up, then people could swap
in open source. Now, I'd love to hear
your perspective on this because one of
the things people have talked to me
about when it comes to the anthropic
business is there is that risk
eventually that open source gets good
enough that you can take anthropic out
and put open source in.
>> Yeah. So, you know, people have I you
know, I think one of the things that's
been true of this industry is that and
you know, I saw it early in I saw it
early in the history of AI. Every
community that that AI has gone through,
it has this set of heruristics about how
things work. like back when I was in you
know AI back in 2014 there was an
existing kind of AI and machine learning
research community that like thought
about things in a certain way and we're
like this is just a fad this is a new
thing this can't work this can't scale
and then because of the exponential all
those things turned out to be false then
a similar thing happened with kind of
like people deploying AI within
companies to various applications then
there was the same thought in the
startup ecosystem and I think now we're
at the phase days where kind of the
world's business leaders like the
investors and the business they have
this whole lexicon of commoditization
um you know modes which layer is the
value going to going to which layer is
the value going to acrew to and open
source is this idea that you can kind of
see everything that's going on you know
that that it has a significance that it
kind of undermines
um uh uh the the fact that it you know
the idea that it undermines business and
I actually find as someone who didn't
come from that world at all, who never
thought in terms of that lexicon. This
is one of these situations where not
knowing anything often leads you to make
better predictions than kind of the
people who have their way of thinking
about things from the last generation of
tech. Um, and I, you know, this is all,
I think, a long-winded way of saying I I
don't think open source works the same
way in in AI that it has worked in other
areas. primarily because with open
source you can you can see the you know
you can see the source code of the model
here we can't see inside the model um
you know it's often called open weights
instead of open source to kind of
distinguish that but a lot of the
benefits which is that many people can
work on it that it's kind of additive it
doesn't quite work in the same way um so
you know I've I've actually always seen
it as a red herring when I see it when I
see a new model come out I don't care
whether it's open source or not like if
we talk about deepseeek I don't think it
mattered that Deep Seek is open source.
I think I ask is it a good model? Is it
better than us at at you know the things
that that's the only thing that I care
about it. It actually it actually
doesn't doesn't matter either way. Um
because ultimately you have to you have
to host it on the cloud. The people who
host it on the cloud do inference. These
are big models. They're hard to do
inference on. And conversely, many of
the things that you can do when you see
the weights um uh uh you know, we're
increasingly offering on clouds where
you can fine-tune the model. You can you
know um you know we're even looking at
at ways to you know to to kind of you
know investigate the activations of the
model as part of like an
interpretability interface. We did some
little things around steering last time.
Um so I think it's the wrong axis to
think in terms of when I think about
competition I think about like which
models are good at the task that we do.
Um I think open source is actually a red
herring.
>> But if it's free and cheap to run
>> it's not free. You have to you have to
you have to you have to run it on
inference and someone someone has to
make it fast on inference.
>> All right. So I want to learn a little
bit more about Daario the person. Yes.
So we have a little bit of time left. Um
so I have some questions for you about
early life and then how you became who
you are. Yes.
>> So, what was it like growing up in San
Francisco?
>> Yeah. Um I, you know, the city when I
first grew up here had not really had
not really gentrified uh uh that much.
You know, when I grew up, the tech boom
hadn't hadn't happened, uh hadn't
happened yet. Um you know, it happened
as as I was going through high school.
And actually, I had no interest in it.
Um it was totally it was totally boring
to me. Um you know, I was interested in
being like a scientist. I was interested
in physics and math and you know the
idea of like you know you know like
writing some website actually had no
interest to me to me whatsoever or like
founding a company like those weren't
things that I was uh that I was
interested in at all. Um you know I was
interested in discovering fundamental
scientific truth and I was interested in
like you know how can I how can I do
something that like makes the world
better. Um uh so so you know that was
that was kind of more and you know I
watched the tech boom happen around me
but I I feel like you know there was all
kinds of things I probably could have
learned from it that would have been
helpful now but I just actually wasn't
paying attention and had no interest in
it even though I was like right at the
center of it.
>> So you were the son of a Jewish mother
>> Italian father that is true
>> from where I'm from in Long Island. We
call that a pizza bagel.
>> A pizza bagel. I've never I've never
heard that term before.
>> So what was your relationship with your
parents like? Yeah, I mean, you know, I
was I was always uh I was always I was
always pretty close with them. You know,
I feel like they gave me a sense of, you
know, of of kind of right and wrong and
what was important in the world. I feel
like, you know, kind of imbuing a strong
sense of responsibility is is maybe the
thing that I remember most. you know,
they were always people who felt that
sense of responsibility and, you know,
wanted to wanted to make the world um uh
wanted to make the world better. And and
I feel like, you know, that's one of the
one of the main things that I that I
learned from them. You know, it was
always a very a very um a very loving
family, a very caring family. I was very
close with my sister Daniela, who of
course became my became my became my
co-founder. And you know, I think we
decided very early that we wanted to
work together in some in some capacity.
I don't know if we imagined that it
would happen, you know, at quite the
scale that that that that it has
happened. Um uh but it um you know, I
think I think it really um you know,
that was that was something we kind of
decided early that we wanted to do.
>> The people that I've spoken with that
have known you through the years have
told me that your father's illness had a
big impact on you. Can you share a
little bit about that?
>> Um yes. Yes, he was. Um yeah, you know,
he was ill for a long time. Um and uh
eventually uh uh died in uh eventually
died in uh in in 2006. Um uh so that you
know that was actually one of the things
that drove me to you know I I don't
think we mentioned it yet in this
interview but before um you know before
I went into AI um you know I went into
biology. So, you know, I'd gone to uh I
I you know, I'd shown up at I'd shown up
at Princeton um uh wanting to be a
theoretical physicist and you know, I
did some did some work in in cosmology
for the first few few months of my time
there. um you know and and you know that
was that was around the time that that
my father died and you know that did
have an influence on me and kind of was
one of the things that convinced me um
you know to to go into biology you know
to try and address um uh uh uh you know
human illnesses and biological problems
and so I started talking to some of the
folks who worked on biohysics and
computational neuroscience in the
department that I was at at Princeton
and that was what led to the switch to
biology and computation neuroscience and
then you know of course after that I
eventually I eventually went into AI and
the reason I went into AI was actually a
continuation of that motivation which is
that um you know as I spent many years
in biology I realized that the
complexity of the
underlying problems in biology felt like
it was beyond human scale. you know, in
order to understand it all, you needed
hundreds, thousands of, you know, human
researchers and, you know, they often
had a hard time collaborating or sharing
their, you know, combining their their
knowledge. And AI, which was, I was just
starting to see the discoveries in it,
felt to me like the only technology that
could kind of bridge that gap, could
bring us beyond human scale to, you
know, to to fully understand and solve
the problems of biology. So, yeah, there
is a through line there,
>> right? And I could have this wrong. Uh
but one thing I heard was that his
illness was um largely unccurable when
he had it. And there have been advances
that have been Can you share a little
bit more? Yes. There are advances that
have made it much more manageable today.
>> Yes. Yes. That is uh that is uh that is
that that that is true. actually
actually only um uh uh only in the um uh
maybe 3 or four years after he died the
the cure rate for the disease that he
had went from uh went from 50% to uh uh
uh to roughly 95%.
>> Yeah. I mean it has to have felt so
unjust to have your father taken away by
something that could have been cured.
>> It it of course of course um but it also
tells you of the the urgency of solving
the relevant problems, right? that um
you know that that that you know there
there was someone who worked on the cure
to to this disease that you know managed
to cure it and save a bunch of people's
lives but you know could have could have
um saved even more people's lives if if
you know they they had managed to to
find that that to find that cure you
know a few years earlier than they did.
Um, and I think that's that's one of the
tensions here, right? That, you know,
um, I think AI has all of these
benefits. Um, and, you know, I want
everyone to get those benefits as soon
as possible. You know, I probably
understand, you know, better than almost
anyone how urgent those benefits are.
Um, and so I really understand the
stakes. You know, when I speak out about
AI has these risks and I'm worried about
these risks, I get very angry when
people call me a doomer. I got really
angry when, you know, when when
someone's like, "This guy's a doomer. He
wants to slow things down." You you
heard what I just said, like, you know,
my my father died because of, you know,
cures that, you know, could have could
have happened a few years later. I
understand the benefit of this
technology. When I sat down to to to
write machines of loving grace, you
know, I wrote out all the ways that
billions of people's lives could be
better with this technology. Some of
these people, some of these people who
on Twitter, you know, cheer for
acceleration. I don't think they have a
humanistic sense of the benefit of the
technology. Their their brain's just
full of adrenaline and and they're like
they want to cheer for something. They
want to accelerate. I don't get the
sense they care. Um and so when these
people call me a doomer, I think I think
they just completely completely lack any
moral credibility in doing that. Um uh
you know, it really makes me lose
respect for them. And I've been
wondering what this this word impact has
uh been because it's come up so often
that those who have been around you have
said you've been singularly obsessed
with having impact. In fact, I spoke
with someone who knew you well who said
you wouldn't watch Game of Thrones
because it wasn't tied to impact that it
was a waste of time and you wanted to be
focused on impact.
>> Actually, that's not quite right. I
wouldn't watch it because it was so
negative sum people were playing playing
such negative. It was like these people
start off and they're partly the
situation and partly because they're
they're just horrible people. They like
create this situation where at the end
of it everyone is like worse off than
everyone was before. Um I'm I'm really
I'm really excited about like creating
positive some situations. Um
>> I recommend you watch it. It's a great
it's a great show. But I hear I hear
some parts of it I was just very
reluctant and didn't watch it for a long
time.
>> Let's get back to the impact.
>> Let's get back to the impact. So that's
what impact is is effectively your
career has been this I this quest to
have that impact to be able tell me if
I'm going too far to prevent other
people from being in similar situations.
>> I you know I I I think I you know I
think I think that's a piece of it. I
mean you know I I have looked at you
know many uh you know many attempts to
help people and you know some of them
are more effective than others. Um, and
you know, I think I think I've always
tried to, you know, there should be
strategy behind it. There should be
brains behind um, you know, trying to
trying to help people. Um, you know,
which often means that there's a long
path to it, right? It can run through a
company and, you know, many activities
that are technical and not immediately
tied to the the kind of impact impact
that you're trying to have. But, you
know, the the the the arc is I'm always
trying to bend the arc towards that. I
think I think that's my that's my
picture of it. That's that's really why
I that's really why I got into this,
right? You know, um I think you know,
similar to the reason to get into AI was
that um you know, I I saw the problems
of biology as as almost intractable
without it or at least too slow moving.
Um, you know, I think my reason to start
a company was that I had worked at other
companies and I I I I just didn't feel
like the way those companies were were
run was was really oriented towards, you
know, trying trying trying to have that
impact. There was a story around it that
was often used for recruiting, but it
became clear to me over the years that
story was not sincere.
>> I'm going to circle around a little bit
because it's clear that you're referring
to OpenAI here. Um,
from what I understand, you had 50% of
OpenAI's compute. I mean, you ran the
GPT3 project. So, if anyone was going to
be focused on impact and safety,
wouldn't have been you.
>> Uh, yes. I I was, you know, there was a
period during which uh during which that
was uh that was true. That wasn't true
the entire time. That was, for example,
when we were scaling up GPT3. Um, yeah.
So, you know, I I when I was at OpenAI,
I and a lot of my colleagues, including
the people who, you know, eventually um
eventually founded Anthropic,
>> the pandas,
>> um the pandas. Um
>> that's the name you gave them.
>> I that that isn't a name I gave them.
>> The name they took.
>> Uh that isn't a name they took. Um
>> that's the name other people called
them.
>> Uh I I maybe it's a name other people
called them. Uh that's not a name I ever
used for my team.
>> Okay. Sorry. Go ahead.
>> Uh
>> that's good clarification. Thank you. Um
uh so uh yeah um you know we were
involved in scaling up these models
actually the original reason for
building GPT2 and GPT3 it was an
outgrowth of the kind of AI alignment
work that we were doing right where
myself and Paul Cristiano and some of
the anthropic co-founders had invented
this technique called RL from human
feedback um uh and that was designed to
help steer models in um uh you know in a
direction to follow human intent. It was
actually a precursor to you know we were
trying to scale up another method called
scalable supervision which I think is
just starting to to to work many years
later to help models follow more kind of
scalable uh uh uh human intent. But what
we found is even with the more primitive
technique RL from human feedback it
wasn't working with the small language
models with you know GPT1 that we
applied it to and that had had been
built by other people at OpenAI. So the
scaling up of GPT2 and GPT3 was done in
order to kind of study these techniques
in order to apply RL from human feedback
at scale. Um uh you know this goes to
one thing which is that I think in this
field the alignment of AI systems and
the capability of AI systems is
intertwined in this way that always ends
up being kind of more tied and more
intertwined than we think. Um actually
what this made me realize is that it's
very hard to work on the safety of AI
systems and the um capability of AI
systems separately. It's very hard to
work on one and not the other. I
actually think the value and the um way
to inflect the field in a more positive
way comes from organizational level
decisions. when to release things, when
to study things internally, what kind of
work to do on systems. Um, and that was
one of the things that kind of
motivated, you know, me and some of the
other, you know, to be entropic founders
to kind of go off and do it our own way.
But again like if you were driving if
you think language uh if you think
capabilities and safety are interlin and
you were the guy driving the cutting
edge models within open AI you know you
if you left you knew they were going to
be a company that was still doing this
stuff. That's right.
>> It seems like if you're driving the
capabilities you'd be the one in the
driver's seat to help it be safe the way
that you wanted to. Again, I will say um
you know if there's a decision on
releasing a model, if there's a decision
on the governance of the company, if
there's a decision on you know how the
personnel of the company works, um you
know, how the company represents itself
externally, the decisions that the
company makes with respect to
deployment, the claims it makes about
how it operates with respect to society.
um you know many of those things are not
things that you control just by just by
uh training the model and you know I
think I think trust is really important
I think the leaders of the company of a
company they have to be trustworthy
people they have to be people whose
motivations are sincere no matter how
much you're driving the the forward the
company technically if you're working
for someone whose motivations are not
sincere who's not an honest person who
does not truly want to make the world
better it's not going to work you're
just contributing to something bad.
>> So, and I'm I'm sure you've heard the
criticism from people like Jensen who
say, "Well, Daario thinks he's the only
one who can build this safely and
therefore, speaking of that word
control, wants to control the entire
industry."
>> I've never said I've never said anything
like that. That's an outrageous lie.
That's the most outrageous lie I've ever
heard. Um,
>> by the way, I'm sorry if I got Jensen's
words wrong, but
>> No, no, no. The words were correct.
Okay. the but but but but the words are
the words are outrageous. In fact,
>> I've said multiple times and I think
Anthropic's actions have shown it
>> that um
you know, we're aiming for something we
call a race to the top. Um you know,
I've said this on podcasts over the
years and I think anthropics actions
have shown it where you know with a race
to the bottom, right, everyone is
competing to like, you know, get things
out as fast as possible. And so I say
when you have a race to the bottom, it
doesn't matter who wins, everyone loses,
right? Because you make the unsafe
system that you know helps your
adversary or causes economic problems or
uh you know is is unsafe from an
alignment perspective. The way I think
about the race to the top is that um it
doesn't matter it doesn't matter who it
doesn't matter who wins, everyone wins,
right? So the way the race to the top
works is you set an example for how the
field works. to say um uh uh you know
we're going to engage in this practice.
So a key example of this is responsible
scaling policies. We were the first to
put out a responsible scaling policy and
you know we didn't say everyone else
should do this or you're bad guys. We
didn't you know we didn't you know kind
of try to use it at his advantage. We
put it out and then we encouraged
everyone else to do it. Um and many and
and then we discovered in the months
after that that you know there were
people within the other companies who
were trying to put out responsible
scaling policies but the fact that we
had done it allowed you know gave those
people permission right kind of kind of
enabled those people to um you know to
make the argument to leadership hey
anthropic is doing this so we should do
it as well. The same has been true of
investing in interpretability. we
release our interpretability research to
everyone um uh and allow other companies
to copy it even though we've seen that
it sometimes has commercial advantages
same with things like constitutional AI
same with the the the measure the the
the you know the measurement of the
dangers of our system dangerous
capabilities evals so we're trying to
set an example for the field but there's
an interplay where it helps to be a
powerful commercial competitor I've said
nothing that that any that anywhere ware
near resembles the idea that this
company should be the only one to build
the technology. I don't know how anyone
could ever derive that from anything
that I've said. Um it's it's it's it's
just Yeah. Yeah. It's just it's just
it's just a it's just an incredible and
bad faith distortion.
>> All right, let's see if we can
lightening around like one or two before
I ask you the last one, which we'll have
five minutes for. Um
>> what happened with SPF? Like
>> what happened with SPF?
>> I mean, he was one of Go ahead.
>> I couldn't tell you. I couldn't was what
was the what didn't you answer?
>> I I probably met the guy four or five
times.
>> Um uh so I have no great insight um into
the you know what in you know into the
psychology of SPF or or you know why why
he did things as stupid or immoral as as
as as as
he did. I think the only uh the only you
know the the only thing I had ever seen
ahead of time with uh SPF was uh you
know a couple people mentioned to me
that he was like hard to work with that
you know he was he was like a bit of a
move fast and break things guy.
>> Um and I was like okay you know there's
like plenty of people
>> Welcome to Silicon Valley.
>> Yeah. Like welcome welcome to Silicon
Valley. Um and so I remember saying okay
I'm going to give this guy non- voting
chairs. I'm not going to put him on the
board. He sounds like a bit, you know,
he sounds like a bad person to deal with
every day. Um, but, you know, he's
excited about AI. He's excited about AI
safety. He's, he's, you know, he's a
bull on on on AI and he's interested in
AI safety. So, you know, seems like a
seems like a sensible seems like a
sensible thing to do. you know in in uh
in in you know in like in in in in
retrospect um you know that that you
know move fast and and and break things
you know was turned out to be much much
much more extreme and bad than than you
know than than than I ever imagined.
>> Okay. So let's end here. So you found
your impact I mean you're you're working
the dream pretty much right now. I mean
think about all the ways that uh AI can
be used for biology uh just a start. You
also say that this is a dangerous
technology and I'm curious if your
desire for impact um could be pushing
you to accelerate this technology um
while you know potentially devaluing the
possibility that it could that
controlling it might not be feasible. So
you know I think I have more than anyone
else in the industry warned about the
dangers of the technology. Right? We
just spent 10 20 minutes talking about,
you know, the the frightening the, you
know, the large array of, you know,
people who run, you know, trillion
dollar companies criticizing me for, you
know, for talking about the the the
dangers of these technologies, right?
You know, I have US government
officials. I have people who run $4
trillion companies criticizing me for
talking about the dangers of the
technology, right? imputing all these
bizarre motives that bear no
relationship to, you know, to anything
I've ever said, not supported in
anything I've ever done. And yet, I'm
going to continue to do it. Um, I
actually think that, you know, as the
revenues, as the economic business of AI
ramps up, and it's ramping up
exponentially, you know, if if I'm
right, in a couple years, it'll be the
biggest source of revenue in the world,
right? It'll be the biggest industry in
the world. And people who run companies
already think it. So we actually have
this terrifying situation where uh you
know hundreds of billions to trillions
to I would say maybe 20 trillion of
capitals on the side of accelerate AI as
fast as possible. we have this, you
know, company that's very valuable in
absolute terms, but you know, looks very
small compared to that, right? 60 60 $60
60 billion. And I keep speaking up even
if, you know, it makes folks in, you
know, there have been these articles f
you know, some folks in the US
government are upset at us, for example,
for opposing the moratorium on AI
regulation, for being in favor of export
controls for chips on China, for talking
about the economic impacts of AI. Every
time I do that, I get attacked by many
of my peers.
>> Right. But you're still assuming that we
can control it. That's what I'm pointing
out.
>> But I'm I'm just I'm just telling you
how much how much effort, how much
persistence, how much despite everything
that stacked up, despite all the
dangers, despite the risk that it has to
the company of being willing to speak
up, I'm willing to do it. And and and
that's that's what that's that's why I'm
saying that look if if I thought that
there was no way to control the
technology, right? If I thought even
even if I thought this is just a gamble,
right? Some people are like, "Oh, you
think there's a five or 10% chance that
AI could go wrong, you're just rolling
the dice." That's not the way I think
about it. This is a multi-step game,
right? You take one step, you build the
next step of most powerful models, you
have a more intensive testing regime. As
we get closer and closer to the more
powerful models, I'm speaking up more
and more and I'm taking more and more
drastic actions because I'm concerned
that the risks of AI are getting closer
and closer. We're working to address
them. we've made a certain amount of
progress, but when I worry that the
progress that we've made on the risks
does not you know is not fully aligned
with the um uh you know is not going as
fast as we need to go for the speed of
the technology then I speed up then I
then I speak up louder. Um, and so you
know, you're asking why am I why am I
talk, you know, what, you know, you
started this interview by saying what's
gotten into you? Why are you talking
about this? It's because the exponential
is getting to the point that that I
worry that we may have a situation that
our ability to handle the risk is not
keeping up with the speed of the
technology. And that's how I'm
responding to it. If I believe that
there was no way to control a
technology, which I I I I I see
absolutely no evidence for that
proposition, we've gotten better at
controlling models with every model with
every model that we release, right? All
these things go wrong, but like you
really you really have to stress test
the models pretty hard. That doesn't
mean you can't have emergent bad
behavior. And I think, you know, if we
got to much more powerful models with
only the alignment techniques we have
now, then I'd be very concerned. then
I'd be out there saying everyone should
stop building these things. Even China
should stop building these. I don't
think they'd listen to me, which is one
reason I think export controls is a
better is is is is a better measure. But
if if we got a few years ahead in models
and had only the alignment and steering
techniques we had today, then you know,
I would definitely be advocating for us
to to, you know, to to to to slow down a
lot. The reason I'm warning about the
risk is so that we don't have to slow
down. So that we can invest in safety
techniques and can continue the progress
continue the progress of the field. It
would be a huge economic effort even if
one company was willing to slow down the
technology. You know that doesn't stop
all the other companies that doesn't
stop our geopolitical adversaries to
whom this is a existential fight fight
for survival. So, you know, there
there's there's there's very little, you
know, there's very little latitude here,
right? We're stuck between all the
benefits of the technology, the race to
the race to accelerate it and the fact
that that is a multi-party race. And so,
I am doing the best thing I can do,
which is to invest in safety technology
to speed to to speed up the progress of
safety. I've written essays on the
importance of interpretability on how
important various directions in uh in in
in safety are. We release all of our
safety work openly because we think
that's the thing that's a public good.
That's the thing that everyone that that
that that that everyone needs to share.
So if you have if you have a better
strategy for balancing the benefits, the
inevitability of the technology and the
risks that it face, I am very open to
hear it because I go to sleep every
night thinking about it because I have
such an incredible understanding of the
stakes in terms of in terms of the
benefits in terms of you know what it
can do, the lives that it can save. I've
seen that personally. I also have seen
the risks personally. We've already seen
things go wrong with the models. You
know, we have an example of that with
Grock. And you know, people dismiss
this, but they're not going to laugh
anymore when the models are taking
actions, when they're manufacturing, and
when they're in charge of, you know,
medical medical interventions, right?
People can laugh at the at the risks
when the models are just talking. But I
think it's very serious. And so I think
what this situation demands is a very
serious understanding of both the risks
and the benefits. These are highstakes
decisions. They need to be made with
they they they they need to be made with
a seriousness. And and I think something
that makes me very concerned is that on
one hand we have a cadre of people who
are just doomers. People call me a
doomer. I'm not. But there are doomers
out there. People who say they know
there's no way to build this safely. You
know, I I've I've looked at their
arguments. They're a bunch of
gobbledegook. The idea that these models
have dangers associated with them,
including dangers to humanity as a
whole, that makes sense to me. The idea
that we can kind of logically prove that
there's no way to make them safe, that
seems like nonsense to me. So, so I
think that is an intellectually and
morally unserious way to respond to the
situation we're in. I also think it is
intellectually and morally unserious for
people who are sitting on 20 trillion
dollars of capital who all work together
because their incentives are all in the
same way. There are dollar signs in all
of their eyes um to sit there and say we
shouldn't regulate this technology for
10 years. Anyone who says that we should
worry about the safety of these models
is someone who just wants to control the
technology themselves. That's an
outrageous claim and it's a morally
unserious claim. We've sat here and
we've done every possible piece of
research. We speak up when we believe
it's appropriate to do so. We've tried
to back up, you know, when we make
claims about the economic impact of AI.
We have an economic research council. We
have an we have a we have a economic
index that we use to track the model in
real time. And we're giving grants for
people to understand the economic the
economic impact the the economic impact
of the technology. I think for people
who are far more financially invested in
the success of the technology than than
than than I am to just you know breezily
lob add ad homonym attacks you know I
think that is just as intellectually and
morally unserious as the doomer's
position um I think what we need here is
we need more thoughtfulness we need more
honesty we need more people willing to
willing to go against their interest
willing to not have you know breezy
Twitter fights, uh, hot takes. We need
people to actually invest in
understanding the situation, actually do
the work, actually put out the research,
and and actually add some light and some
insight to to the situation that we're
in. I am trying to do that. I don't
think I'm doing that perfectly as no
human can. I'm trying to do it as well
as I can. It would be very helpful if
there were others who would try to do
the same thing. Well, Dario, I said this
off camera, but I want to make sure to
say it on as we're wrapping up. Um, I
appreciate how much Anthropic publishes.
We have learned a ton from the
experiments, everything from red teaming
the models to vending machine Claude,
which we didn't have a chance to speak
about today. Um, but I think the world
is better off just to hear everything
going on here. And, and to that note,
thank you for sitting down with me and
spending so much time together.
>> Thanks for having me.
>> Thanks everybody for listening and
watching. and we'll see you next time on
Big Technology Podcast.