The Path to SuperIntelligence & Escaping the Simulation — With Nick Bostrum

Channel: Alex Kantrowitz

Published at: 2024-08-07

YouTube video id: a4SyxHnDhgE

Source: https://www.youtube.com/watch?v=a4SyxHnDhgE

why do you believe that it's a certainty
that will sort of achieve artificial
general intelligence or super inell
really certain certainty is too strong a
word I think you feel it's very likely
yeah at least conditional on um Science
and
Technology continuing and like we don't
don't we could have some sort of
civilizational collapse or we could
eventually um you know go extinct or
destroy ourselves in some other way not
unrelated to AI there are like
developments in synthetic biology and
other new weapon systems and
whatnot um
but if if kind of
these development efforts in in in in
hardware and in in in algorithms
continue then it looks very likely that
we will succeed in this I mean from
first principles we have the human brain
as an existence proof that general
intelligence is possible there is no
reason at all to suppose the human brain
is in any sense optimal like neither
from a hardware point of view uh nor
presumably from the algorithmic point of
view um so just as we have machines that
are physically much stronger and faster
than human bodies or any animal body is
then likewise we will have eventually
cognitive systems that are much faster
and clever um so that that's like a very
high level um argument but then you can
also look just at the kind of advances
we are seeing where more and more things
that used to be impossible for to do
have become done uh and um there are
just a lot fewer left of these kind of
uh
Milestones
um there is this phenomenon that like
before it is done it looks really hard
once a I have done it then we kind of
quickly forget how just how like
impressive it was and just take it for
granted like yeah of course computers
can play chess like of course what's the
big deal but like at the time like it
was a big and then you can see like oh
we have they can play go they can see
they can imagine they can write poetry
they can write computer programs they
can talk to us in ordinary language they
like can pass like you know um
undergraduate level exams in all these
different subjects like this this this
is a lot of stuff that like to people 30
years ago would have been like wow uh
you must be really close to ADI if you
have done all of these things you know
there's a philosophy in Silicon Valley
and some people dismiss it but I think
you know they should give it Credence
because a lot of the money is behind it
that it's called effective
accelerationism it's almost like a meme
on on Twitter and um you know I'm
curious what you think basically it says
we should just continue to develop and
not pay any mind to the potential risks
because ultimately just the active
building is good in in and of
itself no I I think we should pay heed
to the uh risks
um um
um yeah people have a tendency to sort
of simplify a complex set of
considerations down to something you
could write on a banner and then you
form a little tribe that that kind of
carries that Banner who who then has an
excuse to sort of uh fight verbal
battles against people from another
tribe to carry another banner and it
seems like a very in inefficient way for
human Collective deliberation to occur
that there are these kind of
like semi idiotic tribes that kind of
hopefully in aggregate kind of represent
all the different views and may maybe it
sort of shakes out so that the relevant
considerations are taken into account
but um it's almost like each person to
the extent that they join the fry in
that way like reduces themselves to
being one neuron in in this Collective
brain and then hoping that it adds up to
sanity whereas I think our Collective
brain is is kind of a bit uh crazy
sometimes right I'll
continue no um
so yeah I I think some of it is also
more like a kind of FR frustration with
the ways
that um other forms of technological
progress uh is is
styed by various culture like kind of
overregulation for example like where
each little regulation that some some
government or the EU passes seem like in
isolation kind of well you know it fixes
this problem but then you end up in the
situation where you have like millions
of regulations and the overall effect is
this massive drag on on just doing
things that could help and eventually
you get stasis and so and and also like
in um amongst people talking about like
in like the ethical uh concerns you see
it in biom medicine and you see it in
other fields environmentalism so there's
often like a legitimate problem there
that people are concerned with but
sometimes people who professionally take
it as their job to identify ethical
problems like become very good at
pointing out what all the reasons are
for not doing something all the problems
and issues but they kind of get it's
harder to sometimes see the the cost of
of limiting ourselves in all those ways
because the new innovations that would
be unlocked are not yet there where it's
like the pollution is is there
immediately right or the the person like
if a self-driving car runs over a
pedestrian like that's immediately
visible but all the people who could be
saved if self-driving is developed and
works you know there is no new story
about somebody who didn't die from a
traffic accident and so if you
generalize that across all the different
uh sectors of the economy and Science
and Technology then it's natural to
build up this sense of frustration that
oh there are these these people who are
sort of strangling the human spirit and
like all these good things that we could
have had if it weren't for these
naysayers and and then you build up like
a kind of uh movement to push against
that I think some some of the impetus
for Effective accelerationism comes from
that kind of cumulative frustration and
then it's applied to AI in particular
and then I mean what they're pushing
back against is effectively I mean
effective altruism right that's the one
that they view themselves as the
opponent of and I mean it's effective
alism emerged from Oxford and you've I
think you're ideas have inspired a lot
of their views their thinking on
artificial intelligence so how have you
viewed the evolution of EA yeah so I
mean i' I've never U described myself as
an effective altruism I think my uh uh
uh some of my ideas have had an in an
influence there and in
particular um yeah try to think more
about the sort of the the macro
strategic aspects of our strivings as as
opposed to just the local mediate
effects of what we do um and so yeah I
think there is like a contingent things
because AI has emerged as a sort of a
very Central cause area for many in the
effective alism
movement because they now recognize that
AI will be a big very impactful thing so
trying to shape that for the better
seems like really important in this day
and age right um and
then more recently a lot of that has
done taking the form of sort of
pessimism about the likelihood of the
alignment problem being solved and and
hence then resorting to the idea well if
we can't solve the alignment maybe at
least we could slow down the AI progress
so that you know we have more
time um and
then slowing down AI progress becomes
then more of sort of political like it's
not like a technical solution you just
like figure some somebody's really ever
and write something on the Whiteboard
and you have the formula right this is
more like something that then requires
activism um and so and but once you
start to engage in
activism
then there is kind of mtic pressures to
to simplify our message and to close
ranks and to try to beat the people with
opposing views down in the marketplace
of
ideas um and I think we're seeing some
of the beginnings of that um where there
are these kind of campaigns uh MH to try
to push a certain view um not not like a
kind of open-minded intellectual inquiry
to try to figure out where the truth
lies like there's like an assumption
we've already figured out the truth and
now the like the the the task is more to
sort of uh ramage down the throat of of
people who have a different View and and
similarly on the opposite side so but
this is not Universal there's like I
think you effective altruism is is like
a broad tent and um um there's like a
range of different views in in that
field and I I wouldn't want to sort
of
um paint the whole field with a uniform
color there U are a lot of uh people who
are very much doing their own thinking
and have very complex uh like World
models of how how this might play out
right okay let's talk about Utopia a
little bit um just give us your
perspective on what could go right in
the besta well why don't we do this way
what could go wrong in the worst case
scenario of AI what could go right in
the best case scenario of AI and how do
how do humans have a uh influence in
terms of which direction we
go well I think I mean at the minimum on
the downside I think existential risks
certainly are part of that picture and
as as well on and there's like a
different community so you mentioned
like the effective accelerationists and
the doomers so those are certainly there
then there are also the people who are
more concerned about sort of the
immediate impact of AI on society um yes
you know discrimination or or censorship
and IP like and those are also
legitimate issue I just want to
acknowledge those even though they are
kind of like a separate node in this but
yeah um so I think there is like the uh
the real X risks existential risks that
will arise as we develop uh and possibly
not even super intelligence but you
could imagine even something short of
that making it very easy to develop new
weapons of mass destruction in using
synthetic biology or other so so are you
more concerned about humans using AI to
hurt each other or AI hurting
us um well I think they are both uh
worth worrying about
um
I I think with the X risks I mean maybe
a slightly larger on the AI uh being the
kind of agentic part there
um um but but certainly uh we we really
need to do a decent job on both uh sort
of the alignment and and the governance
for for us to have a good outcome so so
yeah so now on the upside um like
there's a huge unlock and
and a lot of that is just removal of a
bunch of negatives
um um if if you look around at at the
the world as it is now it's not that
Rosy a picture in many ways it's quite a
horror show uh with people dying from
you know Alzheimer's or kids getting
cancer and like starvation and people
being bombed and like all kinds of or
just at the more mundane level people
spending most of their uh adult life you
know working uh in in a sort of boring
occupation that gives them no
fulfillment but they just have to do it
to you know to pay the rent
and like you know headaches and stomach
aches and like all all kinds of just the
totality of all of this extreme misery
and very common more everyday misery um
that's just within the human sphere and
then you add the animal kingdom with
like um so just kind of ameliorating all
that suffering would already be I think
a very strong argument that something at
some point needs to be done here can't
like go on like this I mean do we want
like another you know 10,000 year 100
thousand years of just this um but I
think on top of that there is like the
potential to
also um unlock like new levels of
flourishing Beyond those which are
possible even under ideal conditions in
the current
world
um that's a lot harder to paint a very
concrete picture of because we are sort
of limited in our ability to imagine and
appreciate just as you know if you
imagine like the great ape ancestors of
homo sapiens kind of thinking about what
could be so good about being you know
human uh and so they
might like realize a few things like oh
we could have banana plantations and
have like a lot of bananas and stuff and
and that is true we can have a lot of
bananas now but there's more to Being
Human Than Just unlimited bananas right
like we have sort of you know music and
poetry and film and humor and romantic
love and like all all kinds of stuff
science uh so similarly there is
probably like if we unlock as it were
the a greater space of possible modes of
being there are some in there I'm sure
that are extremely
valuable um in that that I think AI
would be the most plausible path toward
words realizing MH um
and so that's kind of um
but if if I then really Dives in and
tries to think more specifically about
like what would the best possible
continuation of a life starting from
like our current human starting point
look like
then there are some quite interesting
philosophical questions that that arise
and so this this book deep Utopia it's
not really an attempt to sort of well
before we looked at the downside now
let's make the case of how wonderful the
upside could be I think the upside could
be extremely wonderful but that's not
sort of the thrust of the book it's more
like let's just look at this what would
happen if we actually did succeed in uh
creating a solved World um as I call it
like where all the Practical problems
are already solved or to the extent that
there are problems that are not solved
they are in a way better dealt with by
Advanced AIS and robots than by us um
and there is like some aspect of that
condition that at least Prima fascia
look quite
unappealing to our current uh
sensibilities
we often Define our sense of self-worth
on on the idea of of being a contributor
like you're you're a bread winner or you
make like a positive difference in the
lives of your friends or of society at
large you like you you bring value to
the world um
so much of our existence is kind
of um constructed
within the constraints of various
instrumental Necessities that have been
with us since you know the dawn of the
human species there always been a lot of
things that we need to do just to
survive um and if you remove all of
those um there is at least initially
this sense of kind of disorientation or
an undermining of of the like we feel
like kind of what's the purpose like we
would just be these blobs but this is
different from what we spoke about the
last time you and I were on on the phone
this was in 2019 I think you said that
um we'd have to find some new source
this is from from our conversation that
I put in my book you we'd have to find
some new sources of self work worth but
the in in Disneyland the job of children
there is to enjoy the whole thing and
Disneyland would be a rather sad place
if it weren't for the kids so we you say
we would all be like kids in this giant
Disney Disneyland maybe one that would
be maintained and improved by our AI
Machine Tools so effectively that even
if we didn't have to do any sort of
sustainment work that gets turned over
to AI we could be actually quite
fulfilled in life yeah so how do you get
from there to where you are
today well I think that's basically
correct but we can distinguish two
different uh senses of uh fulfilled um
or of having purpose so there is first
of all what you might say the subjective
sense like the feeling of fulfillment or
the feeling of having purpose like this
like this the the emotion of being
motivated and you're really excited
about what you're doing right like that
kind of psychological state that
certainly you could have that in a in a
Sol World in Utopia the utopians could
have like extreme levels of motivation
and immersion and subjective purpose
that's easy that's like a check
mark um and more broadly you can go
through different possible human values
and for some of you can just write off
the bat say well yeah sure of course
that would be trivially easy to do in
Utopia so in this case through the
psychological uh engineering techniques
that they would have I mean already you
could have like imagine a drug without
side effects and without addiction
potential that that just induced the
state of Fascination and motivation we
already have simple versions of that
like but you could imagine far more
sophisticated ways that would give the
utopians like very fine grained d
control over their mental states and
their psychology that would follow from
technological maturity right um so that
so that that's that's easy now some
people however think that there is also
a more objective concept of
purpose uh where it's not just that you
feel motivated but that what you are
doing is actually objectively worth
doing um um and that that's a little bit
uh less obvious to what extent the
utopians would have
that in as much as at least at at first
sight it looks like anything they could
do they wouldn't have to do because they
could just press a button and the
Machine could do it instead except for
those those few things that you bring
bring up that actually we want humans to
do like for instance like ordain a
marriage right like that is something or
read a poem people might want this
potentially yeah they're like yeah so
you could automate everything but except
there might be certain jobs where the as
it were consumer has a direct preference
that the job be done by human in which
case by as it were almost by definition
it's not automatable exactly and this
whole idea of a solved world is
basically where AI can effectively take
care of everything all of our needs all
the production and we are in this Utopia
because the machines have done all the
hard stuff that we don't want to do
anymore yeah uh so a solved world like
is a condition of technological maturity
coupled with whatever uh degree of sort
of um so you could imagine like a very
dystopian scenario with advanced
technology if you have like a sort of
totalitarian despotic right but imagine
also that to whatever extent governance
problems can be solved uh they have been
solved I know maybe you can't solve
governance but to whatever extent they
are like better and worse in terms of
social political structures imagine like
we get something at the good end of that
combined with technological maturity
then that's basically uh the definition
of ass solid World um sorry I know I
took you on a bit of a tangent there no
so so yeah so like they kind of layer so
you could say first well you could
have like like a simple Utopia might be
a kind of post scarcity Utopia where we
just have abundance of material Goods so
we already if you're fortunate enough to
to live in in like a developed country
you know with a decent education Etc
you're all already pretty close to that
you you might might not be able to have
like the ideal yacht of your dreams like
there are some limitations but if you
sort of plot a line that has like the
starting point Hunter gather and the end
point is
like uh complete post scarcity I think
we are more than halfway
there um like it's a bigger difference
to go from not having enough to eat to
having enough to eat than you know to to
get like a slightly more advanced
version of an iPhone or like mhm like a
third house if you already have two
houses like diminishing returns so so
that's like you could first consider
this concept post scarcity Utopia okay
so then what's a level of as it were
more radical Utopia than that well you
could have a post-work Utopia where not
just do we have plenty but we don't have
to work to produce that plenty so it's
not just that like we work all day long
and then we have a lot of money and we
buy stuff but imagine you had all this
plenty and you didn't have to work is
slightly more radical conception but not
that radical I mean there are already
people who you know are born with a
trust fund or something and they don't
never have to work and they have plenty
you know again there's limits to how big
their palaces could be but like at least
some approximation but I think we can
then go further and consider even more
radical conceptions so I've already
alluded
to um there is
the post
instrumental uh Utopia that you could
have where it's not just that we don't
have to
work like to make money but we also
don't have to do any of the other things
that we currently have to do for
instrumental reasons so if you know if
you're Bill Gates you still have to
brush your teeth uh you still have to
you know do a whole host of things just
in your everyday life to get the
outcomes you want there's like a limit
to how much you can ask uh your
assistant to do or that you could you
know but but in in this scenario like a
lot of those other instrumental
reasons we have for doing things would
also drop out of the picture with like
super Advanced automation technology and
then I think there's like a step further
than that which is a call it a plastic
um Utopia
where we also have complete control over
ourselves over our own bodies minds and
mental States using like Advanced techn
biotechnologies or newer Technologies
think that's going to we're going to
achieve that that's wild yeah but I
think if you consider what would be
possible at technological
maturity which we can at least play some
lower bounce on through kind of um um
theoretical analysis so we we can sort
of estimate what kinds of computational
systems could be built we can see what
kind of molecular uh Manufacturing
Systems are are possible to build in our
universe even though we can't currently
put them together with the tools we have
now we can see that there is like a path
there other things like um cures for
aging and stuff like that like we don't
have them yet but there is no you know
no loss of physics prevent people you
know from from living indefinitely if
you had like repair Technologies etc etc
perfect virtual realities perfect like
ways to manipulate the brain like that
they're better than drugs like there
like a whole you could there this kind
of in fact like a table in the book that
outlines some of the affordances you
would have at technological maturity and
maybe there will be additional things we
haven't yet thought of but at least
these and so they would I think
enable uh us to instantiate this
condition of plasticity where where
human nature itself becomes
malleable um it's crazy so that
means there are further
um um questions about purpose so right
now like if you didn't have to work for
a living like maybe some people would
say well you know maybe I would start
you know going to the gym War to get fit
right like that's uh you can't hire a
robot to you know go and run on the
treadmill on on your behalf but with
plasticity there would be a shortcut to
that you could pop a pill that would
induce exactly the same physiological
anity on the way to there with
OIC yeah yeah exactly that's like uh one
more step in in that direction um and
and so the thing is with super
intelligence like you get the
telescoping of the future so all these
sort of Science Fiction like
technologies that maybe we would develop
if we had like you know 20,000 years for
human scientists who work for it we
probably will have a cure for aging and
perfect virtual reality and space
colonies and all the rest of it right
but um all of that could happen very
quickly if you have super intelligence
doing the research and development so so
you get this kind of telescoping of the
long term um y but yeah so then then
there like a further set of things that
currently fill the lives of people uh
that we wouldn't need to do um including
things we do for fun so maybe some
person say well you know if I didn't
have to work and uh I had like maybe I
would play you know some play golf all
day long like because why why would you
play golf all along well because it's
fun it gives me joy let's suppose
somebody says that well then in this
condition of plasticity that would be a
different an easier way for them to get
joy they would just pop a pill and they
could get exactly the same level of um
subjective well-being as as like a a
beautifully manicured uh Golf Course
could induce and so I can't even imagine
that type world that's
crazy um yeah so it does then require a
fairly fundamental rethink of of what it
of means to be human in in this
radically transformed condition but it
is kind of the Imp
implicit uh teoss of our current
strivings if you think about so we we
try to the little problems come up
like we triy to solve it and then like
there's like another problem that we all
like so oh our food rot let's invent the
refrigerator like oh we get fat let's uh
like invent o zic like oh our cars
pollute let's make cleaner engines like
and but if you kind of extrapolate and
take all of that to its limit then you
would end up in in situation where we
can like do everything with no effort
right that would be kind of the the
limit of
Technology um AI you know the goal of AI
has all along been not just to automate
a few specific things but to provide the
technology that allows us to automate
all tasks right like AI hasn't really
succeeded until all intellectual labor
can be done by machine and and
um so I think it's kind of we don't we
don't think about it like that but if if
you sort of see what all of this effort
is all these Investments we have in
science and technology and our efforts
to make the economic system more uh
efficient to allow kids to learn more in
school like like all of this kind of
adds up to some sort of Arrow of
attempted progress in a certain
direction and you might as well at some
point stop to think what happens if we
actually get
there yes um and then we do end up I
think in this condition of a solid world
uh and the question is whether we're
ready for that so uh Nick are you able
to stick around for another couple
minutes or do you have a hard I could do
a few more minutes maybe okay all right
let's take a quick break and come back
and ask a little bit about what whether
we can handle that perfect world we'll
be back right after this and we're back
here with Nick Bostrom he's a
philosopher the bestselling author of
super intelligence and author of the new
book deep Utopia life and meaning in a
solve world it's out now great book
definitely recommend you pick it up um
so you you basically said if we get to
this perfect world you think right now
we're effectively unfit to inhabit it
and in your book we sort of look at uh
in the early chapters you sort of look
at the fact that we've increased our
productivity but we using it for
consumption rather than Leisure um and
that's concerning to you is that part of
the reason why you think we're not quite
ready for this um so where where where
do we fall short in our preparation for
Utopia um well I think human nature is
kind of forged and evolved under various
conditions including conditions of
scarcity and condition where there are
like instrumental demands on us that we
need to exert ourselves make efforts try
we need to work just to get by in life
this has been true for hundreds of
thousands of years it's still true to
some extent today although with certain
relaxations like for example food is
much less of an issue for people living
in in in uh wealthy countries and
increasingly also for more middle inome
countries where obesity is becoming an
issue um so there you can already see a
little bit of a mismatch like where
we're kind of evolved to live under
conditions of food scarcity and when
that no longer obtains unless we make
adjustments like we kind of balloon in
size and then we need to try to find
fixes for that but I think a much more
profound mismatch between where we
currently are psychologically and
biologically and and our environment
could arise if we sort of suddenly moved
into a condition of a solved World um so
there would need to be some adjustments
in that I think scenario if we wanted to
take advantage of of all the things that
uh would be possible
and what would those adjustments be um
well I think uh for a
start if if we take for example human
hedonic uh well-being which might may be
like one of the easiest things to look
at just like the
subjective state of positive AFF effect
like are you actually enjoying the
present moment does it feel good or or
is it like uh unpleasant and
so this is like a fairly fundamental
dimension of our psychological State
some people think it's the only thing
that matters if you're a
hedonist in the philosophical sense you
think that the only bad is suffering and
and the only good is is is pleasure not
not necessarily in the sense of physical
pleasure but in the broad sense of sort
of positive mental honic tone um so it
looks like we are kind of designed in in
a way where we have
a fairly powerful habituation mechanism
so if somebody's conditions in life
improve a lot often you know maybe they
win the lottery or something right so
they get really happy they win the
lottery but very quickly the sort of
fonic tone falls back to the
Baseline um right because we are not
designed for permanent Bliss uh we are
designed in such a way that our reward
system motivates us to uh produce more
effort at at whatever level like no
matter how good our our situation is we
are designed to always try to want to
make it better and so we only get reward
when things improve rather than when
things are like at a good level to a
first approximation it's not completely
true I think people under better
conditions are slightly happier than
people under worse conditions and maybe
a lot happier if they're like under
really bad conditions it's like
certainly has a sort of permanent uh
negative effect but now if there were no
more opportunities for improvement and
and and there is like no need for our
Like Instrumental efforts that it seems
very stingy that we would still not be
able actually thoroughly to enjoy lives
so maybe we would want to change that
set point so we could all be much
happier all the time um to actually
relish this this future like it would be
sad like if if everything were like
super nice and we were still miserable
there like and then just living like
that for like millions of years and like
not really that that that would see to
be uh very unfortunate so that that
could be one obvious
adjustment um
um there's like a bunch of other things
as well that like you might imagine
upgrades of various human capabilities
our cognitive abilities our emotional
repertoire our ability to connect to
other people obviously physical
health um and then kind of at the
philosophical almost or like um
our overall attitude to life like so the
idea that you sort of conceive of your
selfworth as being based on the uh your
ability to make a
contribution uh maybe needs to be
rethought here like if we can no longer
make contributions there there is n risk
to that I think there are certain ways
in which maybe the utopians could but at
least to first like our opportunity to
sort of help out other people would be
reduced if there is just less misery and
need in the world to begin with like if
you're a doctor and there was no disease
like you need to like find another
occupation or you can't base your self
worth of being a really good and caring
doctor if there's no if nobody's sick
like then you have to rethink that and I
think at the more General level we would
all have to sort of rethink what makes
like a human life um have dignity in in
this condition where we are no longer
really where it's at yeah okay here's
the last one it's kind of wacky uh but I
thought I'd throw it out there and see
what you think if we reach super
intelligence um and we hit this
Utopia do you think and I know you've
spoken a little bit about the potential
of us to be living in a simulation so if
we reach super intelligence do you there
do you think there's a chance that we're
going to crack out of the simulation and
effectively figure out who's running it
if we are in
one um that that that that would be one
um possible uh scenario right if you're
in a simulation the simulation could
just end with nothing or it could be
rerun or you could like enter a
different sort of environment within the
simulation continues but the sort of
virtual environment changes or indeed
you could be sort of uplifted out of the
simulation into like the world of the
simulators so all of those are at least
kind of metaphysically possible
conditional on the us being in a
simulation in the first place so the the
simulation
hypothesis expands the this this the
space of sort of
realistic possibilities and the realist
like the space of realistic
Futures you might think if you're living
just in a simple materialistic Universe
you die that's the end your brain rots
there's no more experience and there's
really not much room for other things in
given like the laws of physics and us
being purely material with no soul etc
etc if you are in a simulation then
there's like a much wider range of
things that could happen that that would
seem perfectly possible given the
assumption I mean if there if we're
there it would be so crazy to just unzip
and poke our heads out behind the thing
so yeah okay the book is uh deep Utopia
life and meaning in a solv world it's by
Nick Bostrom Our Guest today also the
author of super intelligence bestselling
author Nick Bostrom thanks so much for
spending some time with us today I
enjoyed our conversation me too all
right everybody thanks so much we will
see you next time on big technology
podcast