Can We Trust Silicon Valley With Superintelligence? — With Nick Clegg

Channel: Alex Kantrowitz

Published at: 2025-11-19

YouTube video id: RKdxpKaqUr4

Source: https://www.youtube.com/watch?v=RKdxpKaqUr4

You'd want surely politicians to do
exactly the thing you've just said which
is hypocritical is they take the money
and they invite person X from Silicon
Valley Y to the fishing weekend or the
Gulf retreat. You want them Sean then
still to be able to get up on their hind
legs and excoriate those those companies
pressure to them
>> but they big tech they've done nothing
to big tech. The former president of
global affairs at Meta and deputy prime
minister for the UK joins us for a
conversation about how to save the
internet and whether we should trust
Silicon Valley with super intelligence.
That's coming up right after this.
Welcome to Big Technology Podcast, a
show for coolheaded and nuanced
conversation of the tech world and
beyond. Today we are joined by Sir Nick
Kle, the former president of global
affairs at Meta and the former deputy
prime minister of the UK and the author
of this great new book, How to Save the
Internet: The Threat to Global
Connection in the Age of AI and
Political Conflict. It's going to be a
great conversation, Nick. Great to see
you again.
>> It's good to be here. Let's start with
so you spent a number of years advising
Mark Zuckerberg through the tricky um
minefields of running Facebook and being
this sort of global lightning rod
because of the power and influence that
Facebook had. So let's just do a little
thought experiment to begin with. You
are advising Sam Alman um what do the
next 5 years look like? Like what does
open AI have to be prepared for um as it
grows bigger and stronger?
>> Wow, what a question. uh certainly one
of the things which would be top of my
list and it's hardly um uh this hardly
betrays great insight but I think this
issue of um the level of emotional
dependency that people have on these AI
entities as they become more and more
sophisticated and the um psychological
and ethical dilemmas that um will throw
up particularly for vulnerable adults
and most importantly of course for kids
and for teens, I think is just going to
become an issue that is going to grow
and grow and grow just because of the
the the level of personalized intimacy
in this experience is is like no other
we've ever experienced online. Um and I
would um you know I would strongly urge
Sam Alman and his and I they appear to
be taking some steps but I suspect
they'll need to go a lot further uh to
get well well well well ahead of that
and and probably take a more
conservative stance than of course the
commercial imperatives will you know
will be driving them in in in the other
direction. That's the age-old sort of
dilemma for these companies which is
they want to compete with each other
ferociously and experiment and push the
boundaries. But I think when it comes to
you've obviously got a bunch of
litigation going on already u in the sad
case where some you know kids have taken
their own lives and so on. But I I just
think the um the the the the uh
impersonation effect uh emotional and
otherwise of these AI entities is so
dramatically different to anything we've
dealt with before. So that's something
I'd probably put right at the top of my
list because that's one something which
in the world of politics doesn't divide
politicians, it unites them and and
that. So that's for for sure.
>> Wait, can we before we move on, I think
we should just pause and talk about this
a little bit because we've talked about
this a lot on the show. Yeah.
>> Um it seems like opening eye is actually
going in the other direction. They are
enabling. So, so Sam Alman's perspective
is let's let adults be adults
>> and if they want to have not only
romantic attraction or romantic feelings
um partnership with Chad GPT that's fine
but even going to the point where they
are enabling erotic uses uh or roleplay
between people in chat GPT and this this
I think this probably stems from the
fact that people really built
>> deep relationships lots of people built
deep relationships with 40 which was
this old model that OpenAI eventually
did away with and there was such a
backlash
>> that maybe they're responding to um the
demand. So is this is this shortsighted?
>> Well, I think it it certainly is not
sustainable if that decision was made on
the basis that somehow the problem of
over reliance, overdependency, the
effect on um teen mental uh and and and
emotional well-being is somehow fixed.
And I I haven't follow I don't follow
these things quite as closely as I used
to. But my reading of the assertion made
by OpenAI was that they can take this
risk with more edgy content particularly
sort of sexual se sexualized content for
adults which of course is a massive use.
I mean
>> you'll get sex and pornography is always
of course is one of always the leading
leading use cases of any any new
communication technology but
particularly this one. um uh that that
that was now possible because the
problem about the exposure of kids to
experiences which might make them more
vulnerable to um all sorts of harms. Um
that that was somehow fixed and and I'm
not sure if I've seen proof that that
has been fixed. Um certainly my
knowledge of the old world, the old
social med media world suggests to me
that that very sharp distinction between
it's okay to allow adults to have more
edgy content uh because we've somehow
gated the uh the content which um or the
experiences which are shared with um
younger people. That all of course
relies on a pretty watertight
technological solution to how you verify
who falls on which side of that uh you
know that age barrier and certainly in
the old world of social that's still not
fixed. It it isn't I mean I to be fair
there are some states across the US I
think California most recently and
others who I think are finally doing
roughly the right thing which is uh
creating this sort of one and done app
storebased adjudication on age which I
think is much simpler for parents and so
on. So I think they're moving there but
in a pretty patchy way. So I I I just
don't think it's unreasonable
for society at large through politics,
through the democratic process to say,
"Hey guys, like f we get it. You want
you want adults to have a more edgy
experience and you've got you've got
other competitors who are taking bigger
risks and and and you don't want to sort
of be outflanked by them, but let's just
kind of can we just do that once we've
actually sorted out how to keep younger
people um agegated in a in a way that
everybody agrees works?" And that just
is not the case so far. And look, if
there's any if there's any I think some
of the tendency in Silicon Valley and
dare I say it amongst the sort of
podcast and commentary classes of saying
let's learn the lessons from you know
the last 20 years of social media.
They're not I mean what's the phrase
history rhymes but it doesn't exactly
repeat itself. I think sometimes it's a
little overworn that comparison but this
surely is one where the comparison is
relevant. It's like it is so obvious
that everybody, it doesn't matter
whether you're a kind of libertarian
tech bro or you're working for a, you
know, an organization that's trying to
defend the interests of kids. You'd look
back and think, wouldn't it have been
great if everyone had just started
earlier on this journey, which to be
fair now is actually gathering pace as
people are trying to work out exactly
how to provide more age appropriate
experiences to teens.
>> So, you're in the room with Sam Alman,
>> right? So, I'd say that would be number
one. But but I'm just saying like
>> you're going to you're let's say you're
talking this through with him.
>> What do what do you tell him the next 5
years is going to look like if this
erotic use of chat GPT continues to go
the way that it's going?
>> I think if it goes
>> or even romantic, not even erotic.
>> Yeah. I I actually don't have a huge
problem with the idea that adults should
be able to avail themselves of romantic.
I mean, exactly where you draw the line
between romantic and explicitly
sexualized is of course a tricky one.
But I've got I've got no problem with
the idea that adults uh can make their
own judgments in this area and if if
this is something which is is kind of
useful to them or stimulating to them
and so you know so it's a kind of free
I've got no problem with that at all. I
would say to Sam Alman, listen, if you
don't want to spend most of your time
giving evidence in DC litigation in five
years and you actually want to be Yeah.
and you actually want to be you actually
want to be continue to be lorded as a as
a as a generational
uh tech leader.
>> Um I I just kind of like hey this you
know don't be careful what you wish for
because if you rush into this too
quickly without having done the homework
on the difficult stuff and it is
difficult. It is really it is way more
difficult than people say oh why can't
these tech companies just fix everything
for young people it is more difficult
but I think pending the fact that that
or given that the fact that that is not
fixed and that assertion about open AI
is demonstrabably wrong I would say to
him you will regret this because maybe
not now maybe not next year but a few
years time I can guarantee you there
will be a societal backlash it could
actually potentially be much greater
than it was for the social media apps
because the level of intimacy of
emotional dependency is going to be so
much greater. Um so so I would say to
him you know what's the phrase fina
lente you know rush rush or hurry slowly
um would be my my my my counsel him on
this topic in particular.
>> Yeah it's fascinating that you led with
that and you know one thing that I found
is in my life is it's pretty easy to
slide into a relationship. It's tough to
get out of one once you're there. And if
you're a tech company start you're
starting millions of relationships with
your users, it could be tricky to pull
those apart.
>> And also, you know, it's I mean, you you
study and and and know the Silicon
Valley sub culture perhaps as well as
anybody. Um, it's just, you know, these
guys are tech leaders and they're all
guys. They're all they're tech leaders.
They're they're extraordinarily
accomplished technologists,
entrepreneurs. They're all highly highly
competitive with each other. They're not
relationship experts. They're not
politicians. They're not philosophers.
They're not ethicists. I sometimes sort
of I sometimes think that because
they're so brilliant at what they do in
the commercial and technological field.
We kind of think they're going to arrive
at the right judgment on some of these
other things. They're not. And we
shouldn't expect them to, and we
shouldn't be surprised if they don't.
Which is why I think on things like
that, you know, particularly this issue
of what is appropriate for adults and
what is appropriate for non non adults
and how do you make that that
distinction work, it's kind of we
shouldn't be waiting for the tech
companies to to to decide on that. And
and I think it's actually a good thing.
It's messy. It's messy because it
creates such an erratic regulatory
environment. But I actually think it's a
pretty good thing that the some of the
US states, frustrated as they are, as I
think many people are, that there's so
little action in DC, are starting to
take some of these matters into their
own hands.
>> So, we'll get back to some more stuff
that's coming down the line for AI. But
this is a good moment to pause and think
about the strategy of your former
employer, Meta. Um, because Meta, Mark
Zuckerberg, they've put billions and
billions of dollars into trying to build
personal super intelligence, uh, AI
friends. You know, I've been I was
reporting in Meta 10 years ago, back
when it was Facebook, literally 10 years
ago, 2015, and people within the company
were talking about how they wanted to
build an AI friend. Yeah. And
>> is it that is it that the company sees
that this application of an AI friend
will be so compelling to people that
they may spend want to spend more time
with it than their human friends and
that's why they want to go down this
route. So the conversations I had when I
was still working in Silicon Valley uh
with with folk in Meta and elsewhere um
was interesting because I
you know I have a I'm I'm not a I've
never written a lo line of code myself.
I'm not an engineer. I don't pretend to
be. So I I I always asked lots of
slightly dumbass questions. Um, and I
was sort of I'm old enough and was
senior enough just to ask dumb ask
questions and people would would bear
with me and sort of sort of explain to
me and I would say so what what is this
friend thing like? How how is it a
friend? And I would sort of get I was I
remember being told no just relax. It's
kind of like you know kids have got they
they've got deep relationships with
their teddy bears with their pets with
obviously with celebrities. People
project themselves onto celebrities in
an extraordinary intense way. It's kind
of it's kind of cool that if in the
future, you know, you might have, you
know, your the teen might have seven
best friends and three of them might be
human and four of them might be
humanoid. Uh, you know, AI or or maybe
the ratio is different.
And I got this made me think because I
don't know about you. I my I have I mean
friends are probably more important to
me than I think I think a life which is
rich in friendships is one of the it's
one of the greatest sort of defining
features of a a life well led. And I've
got some dear you know I've got some
deeply deeply sort of close friendships
which I've had during my whole life. Um
and actually when I think about my some
of my friends are really annoying
sometimes. They're kind of really
they're total pain. I love my friends,
but sometimes God, they can be an
absolute B. But what I mean is that
friendship at at a human a profound
human level is is a constant act of
compassion and compromise of of of
empathy, of joy, but also of irritation
because we have to work around each
other and we all go through ups and
downs in life and so on. And I realized
that actually what what they were
talking about when they talk about
friends, it's not friendship, it's not
friends at all because you're not really
having to adapt yourself. the entity,
the AI entity is entirely adapting
itself to you.
>> So my my fear, but it's a it's a
slightly intuitive one, is you're not
talking about friendship, which is a
complicated thing where you have to have
the emotional maturity to try and
understand someone else's perspective
and put your own feelings aside for a
minute and prioritize them and all that
kind of stuff, which is the absolute
heart of friendship and so important to
be a adult, to be a well-rounded adult
that you realize that your life is not
all revolving around you, it's also
around your friends and so on. I
suddenly, wow, these things they're not
it's not going to be they're friends as
service. And that's that that worries me
a bit because it doesn't worry me on a
technological level. It worries me on a
human level because I think that could
foster immense narcissism. Oh yes.
>> And sort of neediness and this sort of
expectation that your friends are always
going to be there for you sort of 24
hours a day in exactly the same, you
know, fresh voiced way. And and so I
just I just kind of and remember the I'm
sure the these are very smart people who
are who are working on this in Silicon
Valley. I'm sure the debate has moved
on. But certainly when I first started
asking questions about this some years
ago when I was there when I also heard
exactly what you suggested wouldn't it
be great if particularly and of course
then what what you what the what folk do
they always take the most extreme or the
most heart heart-wrenching example
someone who's completely lonely and
hasn't got friends and of course who's
going to deny that's great if they can
find companionship
or as we've already seen they can um
unburden themselves for mental health
purposes or if they're dealing with
post-traumatic stress disorder and so
on. And I'm not denying any of those use
cases. In fact, I think it's very I'm a
big advocate, I think, for some of these
for AI in the use of mental health, for
instance, certainly to triage basic
conditions. But to make a claim that it
is it is on a par with the complexity of
the give and take of human friendship I
think displays an extraordinary Achilles
heel in the kind of basic philosophy of
um some technologists that somehow
because that isn't friendship that is
friendship of service fine call it
something else call it a companion call
it an assistant call it an aid but don't
pretend it has the richness that true
human relationships which as I say are
often as they're often as infuriating as
they are uplifting. Um, and I certainly
would would um pause a little bit if I
was to if I was to think that future
generations were going to uh rely on
this sort of ontap anxiousness that you
get from from from AI entities. I'm not
sure if that's the best way to to raise
kids to to understand the human
condition.
>> Yeah, I don't think it's a great way
either. We had there are good
applications like we will hear good
stories of the applications. We had the
replica story uh CEO here and she said
that you know she'd been invited to
weddings uh between people and their AI
assistance and by the way built on like
previous generations technology. So you
can only imagine that's going to
continue. But I think one story that she
told that stuck with me was that
somebody who had been through a really
rough divorce started uh said basically
swore off dating humans had formed a
relationship with a replica uh
counterpart or companion whatever you
want to call it AI friend or more than
that and that AI friend basically gave
this person the confidence to start
dating again and they started dating
humans again and they have a human
partner. Okay, but I want to ask this
one question.
>> These are great stories and we shouldn't
deny those but yeah
>> I I'm I'm with you 100%. It's just like
we'll hear those from the tech
companies. We won't hear the other side.
>> Uh but I just want to uh hammer down on
one more thing. Not hammer down just
touch on one more thing and then we can
move on. Um
>> the just from a strategy standpoint.
Yeah.
>> I wanted to get like the product your
perspective for product
>> is is this going to be is meta thinking
that this is going to be such a popular
product u you know that it that it need
that that open AI will threaten it in
this way?
>> Yeah, I I don't know is the answer. I
genuinely don't know. I've been there
for a while. Um I clearly think it is in
the DNA of Meta to believe that it as a
company demonstrabably does has a kind
of
>> um handle on the social aspects on the
the sort of the the the the way in which
people develop intense relationships by
way of and and and with um increasingly
uh online uh experiences. Um so that's
kind of that's in their kind of DNA. Um
uh what I just don't know and I think is
again I'm now talking as a interested
outsider. I genuinely don't I genuinely
don't um is clearly they're throwing a
huge amount of money at both talent and
infra to um compete at the very uh edge
of the best frontier models. Um they've
also got this you know fast expanding
wearables business um which of course
will be digesting a huge amount of sort
of sensory data which is very very
important as these models evolve from
large language models to something far
far more based on visual and sensory
data. So they've got they've got they're
assembling uh the remarkable ingredients
to deliver very powerful experiences.
It's it's not entirely clear to me
whether what actually in the end will be
will happen is that the existing menu of
apps and services that Meta delivers are
just going to massively improve as they
already are for advertisers. I mean if
you look at the AI tools have been used
for advertisers or to your point is it
is it also going to branch into robotics
and and AI you know friendships and so
on. Um my experience of Mark Zuckerberg
isn't it's one of his admirable
qualities. he'll he'll throw everything
at everything. He'll just he'll just and
then he's very very adept at
experimenting with extraordinary speed
of saying that works, that doesn't work.
So I suspect that's the way they're
going to that I think that would be in
keeping with the the the sort of
ambitious philosophy of the of of the
company. But it's just very difficult
for me at this stage to know which one
is actually really going to sing, if any
of them are really going to, you know,
fly. Um, it's clearly going to it's
clearly going to do a tremendous amount
for the existing chassis of of of meta
products. I mean, it's going to lift all
of those boats. It um and I'm sure that
AI entities, companions, friends will
definitely be part of the menu. How
successful or good it will be, how much
people will actually trust them, whether
they will navigate the issues we've just
um talked about um in in a thoughtful
way, well, we'll see.
>> Okay, I'm going to answer my question.
Yes. Uh I think it is a ma major uh
competitive threat for Meta. I think
Meta is the time the AI friends AI
companions uh meta is a time spent
company that is what people care about
there time spent engagement growth of
products
>> if this technology keeps going the way
that it's going uh the AI friend will be
like the stickiest
>> tech product and that to me I think is
something that they're
>> paying close attention to. I'm sure. I'm
sure because because as you as you know
in the sort of the what I'd call the
legacy business is extraordinary way to
describe something which is used by four
billion people and you know still is
generating you know revenue hand over
fist but anyway what's called the legacy
business of course interestingly is
becoming less distinct from its
competitors. So um you know when I
arrived at at as Facebook as it was then
the thing that I always found very
interesting was actually the fact that
it was technology which humans could use
to communicate with each other um share
content which humans had created to
express themselves and so on now and you
see it particularly on Instagram that
the whole thing has shifted more and
more and more to what's he called in the
jargon unconnected content in other
words content that you're seeing which
is being recommended to you
algorithmically from the furthest
reaches of the internet regardless of
whether it has anything to do with you
or your friends or the groups you're on
or so on. And of course, increasingly
content which is rec, you know,
synthetic content which is automatically
uh or was recommended to you by
automated systems. And to that extent,
it's interesting that almost
imperceptibly
uh the meta social media apps are now
competing more and more with Tik Tok and
YouTube. They're becoming they're not
they're not really stages on which
people generate content and communicate
with each other. they are of course
pipelines at which entertainment uh and
entertaining and engaging content is is
is sort of fired at people. So the the
the kind of the the the the market
distinction
uh um of of Meta's existing products is
less distinct from some of those other
major players and it's been for they're
all now roughly in the same ven part of
the ven diagram. So yeah that's that's
new. Yeah. I mean the concept
>> because the social graph saying base
thing was just pretty distinct. It was
distinct and it it gave them an
extraordinary moat that's different now.
>> Yeah. To me the concept of social media
really is dead. You have unconnected
contact content and which they have with
reals. So they're playing there and then
you have messaging groups
>> and they have WhatsApp and messenger so
they're playing there. But this this
first era of you know share with your
friends and your friends are the best
recommener of content
>> to you is
>> Yeah. No. And I mean, listen, you can
you can lament it, but that's the way
that the world's gone. And in a sense,
it's a it's a demonstration of the
extraordinary impact, I think sometimes
even more outsized impact than many
people appreciate of Tik Tok. I mean,
the Tik Tockification of that whole
industry
>> and recommener systems. I mean, that's
an AI thing, too. Once
>> I mean, we're going to talk about China,
I think, coming up, but once
>> people in China got the recommener
system to the point that it was, then
>> it was off to the races from there.
>> That's right. That's right. I mean,
listen, sort of folk like me sort of
lamented a bit because I always I
generally find humans more interesting
than machines. And it I you know, I
always liked the in the old model, um it
was machines that were allowing humans
to be very human um about themselves and
with each other. And and I sort of feel
it's become a much more passive rather
than interactive experience. And I I I
certainly lament that, but you know,
>> no, it's great you brought that up
because I was we spoke 3 years ago at
Davos and uh we were talking about I
ended with this question about the
Mappiness Project which sort of mapped
uh which which activities give people
the most happiness and I mentioned that
social media was like the dead last on
this Mappiness Project and and you
answered a very interesting way. You
said first of all I doubt that you know
three billion people would be that
unhappy that they would come back to
these products every day, right?
>> Uh which is interesting. we could we
won't I don't think we'll spend too much
time on addiction and all that stuff
today. Um but then the other thing you
said was what our research has found is
that passive scrolling actually has less
of or a negative correlation with
happiness where like engagement does.
And it's interesting that like no sorry
the other passive
>> passive passive scrolling makes you less
happy.
>> Yeah. Right. Right. Right.
>> Engaging with stuff makes you more happy
>> which is like intuitively kind of
obvious, right?
>> But all of social media is passive
scrolling now.
>> Well, it's certainly moving in that
direction. Yeah. No, I I um I I uh I
think the passivity of of the experience
is quite different to the more active
and interactive experience of of of
before. But but you know, as you say,
it's also been accompanied by a very
significant shift to much more intimate
forms of communication,
>> right? Messaging apps,
>> messaging apps. And um
>> yeah, and I certainly see in my own
family, my friend group, that's where
people spend a a lot more time. And that
is very interactive. It's highly
interactive. And so, who knows, maybe if
you look at the whole picture, it's not
quite as it's not quite as blunt or as
dismal as kind of active, you know,
active online citizenship to sort of
passive boine consumption of of
recommended content. I think it's way
more mixed than that because people
don't, of course, use one app. They use
multiple apps, particularly young
people. I saw a stat suggest that, you
know, young a young American teen uses
over 40 apps a month. but also this as
you say this extraordinary um growth of
of messaging apps as a as a forum in
which people express themselves. So
maybe you know maybe that's a big way to
offset that that that other trend.
>> We have a very active discord community
uh around this podcast big technology
and I was on it this weekend and I was
just thinking to myself how funny is it
that we effectively the social internet
started with the chat room then it went
to all these platforms we're back in the
chat room. Yeah, we're back in the chat
room. But what does that say about human
nature, right? It just it it says we we
we we've we have an absolutely
overwhelming impulse to communicate uh
to express ourselves and to communicate
with people in settings that we kind of
feel kind of comfortable in and and that
we can kind of visualize um and that is
containable and and and gathers people
in in around similar interests. That
that's not going to go. That's so I mean
that's millions of years of evolution it
seems to me,
>> right? Right.
>> That's anthropology more than technology
>> for sure. And we'll see. I mean, how
many how you're speaking? We want that
comfort. How how much that comfort to
your point is going to be delivered by
people versus
AI friends. That's going to be a big
question. Then back back to sort of
stitching it all together, inevitably
those who might find
>> who might gravitate towards
>> uh AIS for um the vast bulk of their
communication will be those who may just
find it kind of more difficult or
awkward or or to to to communicate in
the you know in your Discord group and
elsewhere. And that then of course
becomes a slightly self- selecting group
of certainly of early adopters.
>> And that's a problem because you would
think that like then they be those are
probably the most influencable people by
technology.
>> Correct. Exactly. Sort of.
>> Exactly. No. No. But I mean I think
that's the nature of early adoption.
It's it's um it's folk who are either
>> open to just just just open to the
ingenuity of new things and or need it.
Yeah. And and and that's that's that is
exactly the the you know one of the
dilemmas. It's one of the reasons back
to your opening question why I think you
need to be super mindful of that because
that that will have a big societal and
political reaction o over time if that's
not handled intelligently.
>> So the big uh AI labs they're filling
their ranks with your former colleagues
is the head of uh consumer apps at
OpenAI. She ran the Facebook app for a
while. Kevin Wild head of product there
former head of product at Instagram.
Yeah, I've met them. I've met both of
them. Mike Kger, who was just on the
show, Instagram co-founder, uh, head of
product at Anthropic. There's many more.
>> Yeah.
>> Um,
>> so I guess I I guess I'm curious to to
hear your perspective on why these
companies have been such a have decided
that they want folks from social media
to take the lead here. Obviously, they
know product, but it's also sort of
>> I don't know if I'm if I'm not getting
the full picture here about what an AI
product, but social media seems like it
wants you to just engage as much as as
as you can because it will show you more
ads where AI is like if it just gets you
to engage for engagement sake, that's
actually like pretty expensive.
>> Yeah.
>> For them to serve. So, what's your
perspective on this?
>> I I my guess is it's much simpler than
that. These are very smart people who've
been in rapidly scaling businesses and
you know if you're if you're Salman or
Darday or any you're going wow I'm
sitting on this I'm sitting on this
rocket ship you know this rocket ship
and it's kind of taking off I need
people around me who understand scale
who can who can um ship products um
quickly uh scale them very quickly and
understand how to operate in complex and
very fast moving uh um environments and
if you if you basically take that as
your one of your list of requirements um
or expectations then of course people
from companies like Meta just you know
feature high up on the high up on the on
the list. No, no, I I I wouldn't have
thought it's it's I wouldn't have
thought it's through the that's my
assumption at least. I wouldn't have
thought it's through the prison that
you've just described, which is one is
engagement with commercial upside, the
other one is engagement with sort of
expensive engagement because I would
have thought at the moment what they're
just racing to do and they're clearly
they're clearly burning a lot of money
in order in pursuit of this objective is
just to is just to expand and get people
using these products. Um, and that in a
sense is a bit of a playbook from from
Mark Zuckerberg. I mean, you know, Mark,
it's one of his it's one of his most
enduring principles, which is build
technology which people find engaging.
You'll work out a way later to monetize
that. I mean, how many years was it that
WhatsApp barely, you know, generated a,
you know, a penny of revenue. And so
maybe that's maybe that's what that's
also something they're they're the the
you know the other the new AI
hyperscalers are are that's the page
they're taking out of out of the meta
playbook.
>> It's just that the ROI for these AI
companies has to be so much better like
ads alone cannot exactly no and but that
but that's the I was about to say $10
million question. It's the it's the
multiple trillion dollar question.
Exactly. that I I and no one seems to
have the answer to that and we're
clearly in this rather odd position
where the the the the you know the cap
the infrastructure investment dwarfs
anything that happened in the run-up to
the dotcom boom.
>> Uh so it's not enough to say oh well you
know this has happened before you know
yeah sure there was a market correction
people went bust a bunch of companies
disappeared but we also had this
wonderful infrastructure that we then
repurposed for other things but this is
just off the scale compared to that. I
mean, I'm sure if you tot up the amount
of hundreds of billions that were spent
uh on on telecom's infrastructure by
some of those, you know, Telos compared
to the
>> I think this year alone will exceed
that. 300 billion plus in capex from big
tech this year and 1 trillion dollars
committed to open AI this year.
>> Yeah, exactly. So, so um and no one's
been explained to me, but I'm not, you
know, I'm I'm not I'm not a financial
analyst, but like no one's been
explained to me how you recoup that that
money. So clearly at some point
something's got to right size,
something's someone's going to lose a
bunch of money. There's going to be a
correction. I I I kind of think that the
the the folk are in the driving seat
here. Um whether it's the new
hyperscalers, nanthropic, open eye,
notably amongst them, or the established
players, Google, Meta, um Amazon, and so
on, Microsoft, I just kind of think
they're locked in a thing where it says,
"Yeah, we don't know where this is going
to go, but we we know one thing for
sure. if we don't compete,
uh, we're sure to lose. So, we don't
know whether we're going to win. We
don't know what the shakeout's going to
be. Um, but the the shest way to lose is
just not to not to throw as much money
as your your next competitor. So, they
are in a bit of a they are in a bit of a
kind of um, you know, spend whatever it
takes mania and and there was a sort of
manic feel about the whole thing. That's
that's that's obvious,
>> right? And so, what they do, they've
told us what they basically need to pay
the money back. It's Mark Zuckerberg's
talked about developing super
intelligence. Sam Alman's talked about
super intelligence and artificial
general intelligence. And this is
>> why does super intelligence equal I yeah
why does that why is that a pot of gold
>> necessarily?
>> If I mean I think this is if you're able
>> Oh, if you're able to hold on to it, no
one else can do it. Well, if you're able
to build technology that's smarter than
every person, the I mean, if you think
about what's economically valuable, that
would be the most economically valuable
thing ever created.
>> If you can hoard it,
>> correct?
>> This is the bit I've never quite
understood. And I may be completely
wrong here, but I'd love to know what
you think. But my I've never quite fully
understood why that would be a hoardable
asset that only one company has keeps
under lock and key and everybody else
kind of everybody else is then thwarted.
Um it seems to me much much more likely
um that it's going to be a more diverse
and dispersed technology than that.
There's a lot of early evidence that
some of the most useful and and um
commercial um models are are are quite
specific ones are small specific ones. I
I I just I don't know until someone
until I at least and this is my rather
sort of primitive way of looking at
these things. Until I have a bit more
a bit more information to be able to
visualize exactly what these slightly
handwavy turns like AGI and super
intelligence really mean. I find it
super difficult to understand why the
assumption is that there is a winner
takes all logic would prevail here.
Maybe it will, but in a world where just
this week, wasn't it? I've forgotten the
what's the Hong Kong based company
that's just produced another uh
particularly good open- source AI model
for agentic and coding purposes. Mini
>> Manis,
>> is that what it is? Yeah. Um M2 or
something.
>> Kim Kim K2. Is that
>> Yeah, maybe. Maybe. Anyway, um you know,
when you've got we're in a world where
you've got now on an where it's now
become like just standard,
it's like become a conventional wisdom
that the world's largest autocracy is
churning out the the the world's most
advanced open source um AI models and
that I see Chesy the other day saying
that Airbnb rely very heavily on Deep
Seek for their own work and so on. I
don't know. I that seems to me to be an
indicator of just how versatile and
dispersed the technology is rather than
how much it can be hoarded in a winner
takes all knockout blow by one of the
labs. But I accept of course a I may be
entirely wrong. Uh but secondly that if
that is the case, if the race to super
intelligence is this eureka moment where
one entity wins, everybody else is left
left, you know, in the dust, then of
course that is of such immense
commercial value. You can hardly put a
figure on it. I get it. I get it. But it
does seem to me to imply a bunch of
fairly heroic assumptions.
>> Okay. So that is a great lead into this
question that I wanted to ask you which
I might have to alter now. I wanted to
ask you whether we can trust Silicon
Valley with super intelligence. Let's
say they do build this all knowing AI
that. Okay. So so let's do that and then
I want to ask you whether they can even
control it itself because of what you
just said. So so let's start with can we
trust them? Super intelligence. Well, um
um I said of course not because in a
sense my knee-jerk response is as I said
earlier these are technology companies.
You um you shouldn't trust technology
companies or should it's not even trust
I I use even a less loaded way of
putting it. It's like don't look to a
technology company to sort out the
moral, societal, political, ethical
tradeoffs
um by which you know which are entailed
in the way in which millions, billions
of human beings interact with
technology. They're technologists.
They're they are hardriving, highly
competitive, highly commercial um
technologist. So So to that extent, no,
that's not their that's not their role.
It's not their it's not their expertise
and it's one of the reasons why this
fashion at the moment certainly in
Silicon Valley and DC this sort of ultra
libertarian thing of any kind of
constraint any kind of regulation is
unacceptable is so foolish because it's
like they don't have all the answers and
nor should you ever expect them to have
all of the the the answers. So so that's
on the on the one that's the sort of
easy bit. the the bit I find um just
harder to answer is um I kind of just
don't yet really know where is what is
the moment we work walk through the
looking glass and super intelligence has
happened. Some people say it's when they
you know when these systems deliver uh
or develop a certain level of autonomy
and an ability to self-improve. Um uh
others claim that in the end actually
there's no way that they can fully
escape the rather clunky probabilistic
um underlying architecture upon which
they're built and they're always going
to come up spit out some slightly
hallucinatory um outcomes. They can't be
fully precise all the time and that
they're never going to entirely es
escape the the chains of human command.
I just, you know, I'm very interested
that there are folk like my dear friend
Yan Lun and others who've been pretty
consistent and sure they get surely get
criticized and mocked for it. But if you
ask yourself who's been most astute in
the kind of commentary of the big trends
of of this technology over the last
three or four years, I would have
thought it's fair to say that people
like him who have claimed right from the
outset, yeah, this is really powerful
technology. It's really versatile,
>> but it's not the only or maybe not even
the best route to um human style um uh
uh machines which really can
self-improve and develop their own
autonomy, their own dare I say it sort
of inverted commas conscience. If that's
true and if in fact that the you know
this alternative paradigm which people
now talk about world models and so on is
is where the future lies then we might
look back at all this all this kind of
super intelligence AGI hype and say wow
that was like that that really was kind
of hand wavy stuff to to to recruit AI
data scientists in Silicon Valley more
than based on something which was fully
realizable. So I I I
um I tend to
always slightly look the other way when
I hear a lot of hype because I think the
hype just becomes it has a
intellectually paralyzing effect. Find
it very difficult to think clearly when
you hear people throw around these kind
of really you know these lofty terms
when I don't know what they mean they
don't seem to themselves have any
consensus about what it means. Um and
where there are very serious folks
saying look the paradigmatic limits of
the techn LMA LLMbased technology is is
going to act as a persistent constraint
on getting there in the first place. So
I I really just sort of feel I just feel
the jury is so out out now but I realize
look there are people in Meta and
elsewhere ex colleagues of mine who I
like and admire enormously who say you
know it's a failure of imagination on
the part of people like me. No, no, this
is this is around the corner and the
scaling laws still prevail and the
relationship between how much you put
in, how much you get out is still holds
and is still surprising us.
>> Yeah, we we we listen to to Yan here
often. He's been on three times. Uh I
think he has a really good perspective.
Um but but let's get back to like the
control part. Yeah.
>> I mean, assuming, you know, let's throw
out the jargon. Yeah.
>> These systems are becoming much more
powerful. Um they're open enough there.
you know the companies have moved
towards closed systems but they're open
enough
>> that something that happens in China can
get copyright into the US and US into
China totally
>> um so is there is there a concern on
your end that like no matter how much
governments or companies might try to
control this technology it's not very
controllable
>> well clearly that is one very real um
possibility if the if the predictions
are even half true about how this
technology might develop its own
>> right
>> its own logic logic its own its sense of
motivations, its own sense of survival.
You know, you've seen these recent
reports from
>> unbelievable stuff, right? They will
manipulate evaluators just to preserve
their values.
>> To preserve their values or or or to
stop themselves being, you know,
eliminated and stuff and to be
extinguished. Um, yeah, of course.
>> I mean, they're hacking they are playing
chess games and then hacking the
program, the chess program to change the
rules so they can win.
>> Oh, wow. No, I hadn't seen that one.
Right. Right. Right. I was referring to
the thing I read.
>> I like when I see the stuff I'm like,
that's so cool and also just a little
bit scary. Yes, it is. It is. It is
definitely um well, it suggests a
survival instinct, which is a very very
kind of profound thought that that they
develop a kind of animallike survival
instinct for themselves. Um um and I
don't want to just simply dismiss that,
but it seems to me at the moment these
are fragmentaryary indications.
They're little sort of flashes of of of
sort of fragmentaryary
evidence that maybe these systems will
um develop fully uh full-blown uh
autonomy in in the sort of sense that we
as humans would would understand. It
seems to me that there's a long long
long long way to go and at the moment
these are being driven by such powerful
systems and so much compute capacity and
so much data. Um is it is it really
plausible to assume that just one more
heave one more you know layer of data
centers um yet more improvements at
inference as well as training levels
will will deliver maybe maybe I I have
to say I'm I'm intuitively a little
skeptical only because I just look at I
ask myself what are the motives for the
folk saying what they say
>> right no I'm not thinking that it's like
an escape scenario I'm just like if you
want this technology to follow a certain
set of values or to be used in a certain
set of
it seeming less and less like that's
possible because of what you're saying
the diffuse
>> it makes it all the more reason it makes
it all the more important and this is
probably the bit where I can speak with
greater authority than I can about the
you know the claim and counter claims
about between different paradigmatic AI
models that's not my expertise that's
why in the end politics does need to
insert itself
>> and that's why this peculiar phase we're
in where DC and Silicon Valley have kind
of which always sort of regarded
themselves with great sort of skepticism
and kept each other at arms length have
fallen into this sort of cloying embrace
with each other and all the tech tech
bros, you know, um in and out of the
White House like like nobody's business.
Um a and married to this very kind of
belligerent America first agenda, you
know, to hell with the rest of the
world. We're in the lead. We're going to
we're going to we're going to we're
going to assert our lead ever more
forcefully and and you know, screw
anyone who says anyone else. Anything
else? I I I would be very surprised if
that is uh a workable strategy for the
US. Um and at some point I think there's
going to be quite a big sort of falling
of the scales from the eyes when they
realize you can't beat you can't beat
you certainly can't beat China like
that. And I suspect in the long run you
can't beat um India either and who knows
about other other places particularly as
they start adopting Chinese open source
models more and more widely in other
parts of the world and then using them
for ever more ingenious purposes. So I
think at the moment we're in this rather
odd phase where we're making where it
seems to me the US political and tech
elite are um united in an assertion
which seems to me to be self-evidently
flawed which is this is not going to
lead to permanent enduring American
supremacy. Um and when that becomes more
obvious, there will need to be a big
course correction and politics in one
way or another in my view which is what
I advocate in the final third of my book
politics ideally in a coordinated
fashion between the world's majest major
you know technocracies the US India and
Europe in that uh descending order of
importance uh will I think at some point
need to rediscover the the the merits of
multilateral action however
unfashionable that is to say in in uh in
you know in the environments in the US.
>> Okay, I want to talk about that a little
bit more and we'll do that right after
this. And we're back here with Nick Kle.
You should check out his new book, How
to Save the Internet: The Threat to
Global Connection in the Age of AI and
Political Conflict. I've read it through
as you can see if you're watching on
video.
>> Nick, I want to talk to you a little bit
briefly before we get into um Silicon
Valley's attachment with the Trump
administration. Um how how Silicon
Valley buys influence in Washington.
There was a terrific story, actually a
great leak of a memo from Brad Smith,
the president of Microsoft. I don't know
if you remember this.
>> I think Microsoft employees were asking
him uh it basically made its way around
the press a couple years ago.
>> Uh the the Microsoft employees were
asking him why Microsoft donates to
packs and causes. And he gave what I
think is like the most interesting and
honest answer. Um I'll just read a bit
of it. he says uh about the money. I can
tell you it plays an important role not
because the checks are big but because
the way the political process works.
Politicians in the United States have
events. They have weekend retreats. You
have to write a check and then you get
invited to participate. So if you work
in the government affairs team in the
United States, you spend your weekends
going to these events. You spend your
evenings going to these dinners. And the
reason you go is because the pack writes
a check. Um but out of an ongoing but
out of that on and out of that ongoing
effort a relationship evolves and
emerges and solidifies and can tell you
as somebody who sometimes uh is picking
up the phone basically to get people to
answer. Yeah. Is that how it works?
>> Yeah. I think it works the way it works
in the US. Yeah. Yeah. It doesn't
absolutely doesn't work like that in the
same way elsewhere. But the U US
political system is so moneyed in a way
that's I think almost without precedent
anywhere in the democratic world
certainly that I know about. Um and in a
sense, you know, what's happened under
Trump too is that that has just become
that transactionalism has just become
way more
overt. You know,
>> you don't like the idea of bringing a
gold bar to the person.
>> Exactly. Gold bars, paying for a gold
laminated ballroom, all the rest of it.
It it's kind of it's kind of like it's
become all like almost like a sort of
pastiche satire of a highly
transactional um cash so money sorry um
financially based um um um you know set
of relationships which I think all the
all the all the big corporates um are
involved in and they do it openly they
do it leg legally they do it lawfully
there's nothing illicit going on um but
in a culture where where elected
politicians are literally non-stop
fundraising
and that's all they're doing. Um, uh,
this has clearly become an established
way exactly as Brad, uh, who I like and
admire enormously. Um, as Brad, um,
explained, it's not it's not you're not
buying a decision. You're you're you're
buying an entry ticket into an event
which then get, you know, etc., etc.,
etc.
>> Retreats fund.
>> I have I I um I didn't go myself. there
were much better placed folk uh in uh in
my meta team who uh who had much better
relationships. You know, I'm a
non-American. My my role was I had
oversight when I was at
>> Meta for the the US operation, but was
was was quite sort of globally focused.
>> How do how do if we talked about in in
the beginning of this conversation sort
of the need for there to be a check on
some of the companies within Silicon
Valley if you know if they're pursuing
this powerful technology. It doesn't
seem to me like there can be if this is
the way the system works.
>> No, I mean it doesn't mean that the US
political system is incapable of
regulating or putting constraints on
companies that uh who whose whose
lobbying teams still make these
contributions to go to the golfing
weekend here or the dinner there and so
on so forth. So I don't I don't I don't
think the evidence is that the US
political system is just rendered um you
know completely inert um by this um you
know by the way in which um corporate
contributions through packs and so on
are are made. um it it just seems to be
the way in which the relationships are
are conducted and established in the in
the first place and and the US body
politic has regulated everything from
the pensions industry to the banking
industry to arms and oil even though all
of those companies and the companies in
all of those sectors are doing exactly
the same thing trying to go to the same
golf retreat going to the same fishing
weekend or whatever it is. Um so so I I
don't think that I think I I um I don't
think it is the case that that means
that um um thoughtful political action
in in in legislative form is made
impossible though I totally understand
why people say or think like you know
your average punters going wow that's is
that the way it works but it is the way
it works in the US. I mean it's not it's
hardly it's hardly a secret and it's
totally open there. There's nothing
illicit about it. That's the crucial
thing. Was it ever weird for you that
these politicians who I'm sure were like
holding these retreats and weekends uh
with your team members were then up on
the um hearing stage and giving the
>> I'm the worst person to ask cuz I you
know I come from a much smaller but but
older democracy called the UK by
comparison and this kind of behan
as I think your wife is German you'll
know a little bit about that you know
it's just I mean
>> it it's just it doesn't mean that there
aren't money issues in the in which uh
um politicians are
>> I just find it so hypocritical that they
would take the money and then call this
tech CEO in for the hearing and lambast
them to show that they're tough on big
tech.
>> But wouldn't you prefer them at least to
do that than then not lambast them. I
mean I think I think I think if anything
>> I prefer them to be effective I guess is
my
>> Yeah. No, no, no. I'm I'm with you and
and um personally as a as a as I say as
a non non non-American
um I'm just stunned at the the the sort
of the extraordinary amount of money
that slloshes around in American um
democracy. But it always has. It's kind
of the way it is. and and the kind of
European approach of having any kind of
state subsidy or anything for ads or for
any kind of politics is is considered to
be for perfectly good reasons perhaps is
considered to be deeply deeply sort of
suspect here in the US.
>> We do it in New York actually in New
York we have matching funds. So if you
every dollar you raise up until a
certain amount you get like seven or
eight dollars. Okay. So
>> well I defer to you on that but that may
be the case but
>> given it is the way it is and right
>> certainly in my job at Meta I dealt with
the way the world that as it was not as
I might ideally want it to be if I could
architect it from its from its
foundations. Um you'd want surely
politicians to do exactly the thing
you've just said which is hypocritical
is they take the money and they invite
the invite person X from Silicon Valley
Y to um the the fishing weekend or the
golf retreat. You want them surely then
still to be able to get up on their hind
legs and excoriate those those companies
and apply pressure to them. But they've
done nothing big tech. They've done
nothing to big tech. They know nothing.
And that's what I'm saying. You of
course you you don't want them to like
say you free pass. But to me the point
is
>> the thing that's really surprising the
system itself is broken if that's what's
going to happen because we see the
effect. And I'm not here saying we need
to have big tech you know massive big
tech regulation. Obviously that could be
misguided as well. It's just that when
you when you look at the way the system
works, it's it's crazy to me.
>> So, my experience just to just to do
something very unfashionable, which is
to stand up for the political class and
to stand up for the um you know, the
really um in many ways really
thoughtful, good people who I saw were
in the in the business of trying to
represent these companies in DC and my
team and elsewhere. They're not they're
not shady. These are really decent
people trying to just do a good decent
day's job. But here's the thing, right?
Um, I often found that the reasons why
there there was no consensus across
across the aisle on issues had less to
do with who's gone on their their
fishing retreat or their golf weekend,
but more to do with deep differences
between Republicans, Democrats on state
preeemption, for instance. You know,
that was something which constantly
bedeled and and stopped uh progress.
Now, you might say that was an alibi for
people not to take action, but I was
always very struck, you know, cuz I
would ask myself, why on earth does the
US of all countries not have a federal
privacy law? It's like that, you know,
this is you would have thought that' be
entirely in line with the constitutional
principles of this country and and I
think it it often came down to very
deeply held views about the relative um
roles and responsibilities of the
federal government and states and so on.
Um the thing I am surprised about I have
to say is why there hasn't been despite
all the energy that that is generated on
both sides of the aisle on this why
there hasn't been more progress at
federal level on um legislating uh to to
to you know to make sure that kids and
teens right are protected in a way that
other users you know which is quite
special compared to other users of
social media and other other um online
experiences that that does seem it and
it doesn't surprise me at all that other
other states It's California is only the
latest example, perhaps the most
significant one. And now taking matters
into their own into their own hands
because that's a reasonable thing to do
if nothing happens in DC.
>> Totally. All right. I have one last one
for you. We have like three minutes
left, but I think this is important. Um,
so at Trump's inauguration, we saw Mark
Zuckerberg, Tim Cook, Sundar Pachai,
Elon Musk.
>> I'm probably missing one. Bezos was
there.
>> Sergey Brin
>> Sergey. So the US tech elite have tied
themselves to the Trump administration
in a way that
>> we haven't really seen before. So I mean
the te tech they loved Obama. Um they I
don't think they were like this close.
Uh so eventually politics goes in
cycles. The Trump's way of doing things
may not be the way that the US wants to
do things forever or the globe wants to
do things forever. What do you think
will end up what do you think will be
the end result for these tech leaders
from tying themselves so closely to
someone who has gone out globally and uh
you know taken this America first
approach.
>> So I think it's short-term and longterm.
In short term it works for them because
they're all you know they're all driven
by FOMO and fear or a mixture of both.
Uh you know one's person's beating a
path to Marlo. Oh my gosh I better get
on the next plane to do it myself.
They're all worried that someone else is
going to somehow um in this highly
transactional environment, this very
capricious transactional environment
where the sort of Trump administration
looks, you know, from a distance like a
sort of form of kind of
institutionalized sort of gangster
capitalism where, you know, favors are
done to favor, you know, favored
companies and individuals and others are
are are you know, kneecapped. um whether
it's countries um you we just saw over
the last 24 hours Canada experiencing
10% increase in a tariff because of an
ad. I mean
>> I mean he'll even do it to to friends
too. Tim Cook has been on the receiving
end and he's again we've talked about
what did the gold bar get him? I don't
know.
>> It's not irrational for them to say wow
this is so random and there are so many
sort of random driveby shootings that
are going on metaphorically I'm speaking
um you know by the by the political
class now. We've just better try and
kind of all do what everybody else is
doing. turn up at these dinners, turn up
at these events, and hopefully we won't
get singled out. Particularly in an
environment where, to our earlier
conversation, they're all spending so
much money on what is to them almost a
commercially um existential race with
each other. Any disadvantage
>> um or any advantage garnered by one of
your one of your uh competitors because
of their relationship with this
administration could be of commercially
of great. So it's not illogical for them
all to do the same thing in the way that
they are. I think the the longerterm
problem is that it just um it just
erodess an immense amount of trust um
across the political spectrum uh you
know in in um in these companies or at
least the leadership of these companies
because if you're a Democrat you're
going wow I remember sitting at a dinner
with you know tech leader X and tech
leader Y saying oh they were great
progressives and now look at them. Uh
but but also honestly I also think from
the Republicans they're going to go I
know that this person said X or Y a few
years ago. So
>> and globally by the way Canada how's it
going to look?
>> Yeah. globally. I mean, you know, you
can imagine what it looks like if you're
in Delhi, Brussels or or or um but I
think many people understand that polit
that that that um business leaders have
to duck and weave, particularly in an
environment where ducking and weaving
seems to be about the only option
available to you in this in this very
capricious um kind of governing paradigm
that you see in the in the Trump 2
administration. I think in the long run
though it of course it poses
difficulties for them. It's trust
eroding in a big way because what
happens if there's a Democrat president?
Well, they're going to suddenly turn
around and turn up at the White House
and say, "Actually, we agree with
everything you've always believed." You
can't do that. So at some point in my
view I as I say I understand why they're
doing everything they're doing but at
some point in the long run all of these
industry leaders if they want to
continue to u see their businesses
prosper for decades to come have to find
some way that they don't go into this I
think slightly demeaning whiplash of
kind of you know
herdlike behavior limpit like herdlike
behavior if I'm not mixing my metaphors
attaching themselves to one
administration then attaching themselves
to another at some point. I hope that uh
a certain kind of distance will be
restored between Silicon Valley and DC.
I don't think it's I often say to people
the about the only worse thing in a
developed capitalist economy than having
major companies and governments at each
other's throats is having them in each
other's pockets. It's much better if
there's a certain wary, respectful
distance between the two. I also kind of
think technological innovation just does
better when it's not too tied up with
the weird vagaries of politics. I and I
suspect Silicon Valley will relearn
that.
>> Well, we could definitely do another
full episode on this topic on what the
values of Silicon Valley actually are
and I hope we get a chance to do that.
But
>> Nick, it's been great to see you again.
You're always welcome on the show. The
book, folks, again, is How to Save the
Internet, the threat to global
connection in the age of AI and
political conflict. All right, that'll
do it for us here, and we'll see you
next time on Big Technology Podcast.