Meta's Chief Technology Officer On Its Augmented Reality Dreams — With Andrew Bosworth

Channel: Alex Kantrowitz

Published at: 2024-10-04

YouTube video id: B0gAe-wXrdQ

Source: https://www.youtube.com/watch?v=B0gAe-wXrdQ

for list I'm sorry I've just put on I
put on a pair of of uh clear and
transparent ones the it starts to feel
like superpowers it starts to feels like
oh like I just have tremendous more
capability than I used to you know so
this idea that we're going to merge with
AI you're not buying it no I don't see
that happening in the near future no I
I'm I'm not saying the you do it if you
could we are getting weird I appreciate
you welcome to Big technology podcast a
show for cool-headed nuance conversation
of the tech world and Beyond and we're
here in men low park with Andrew
Bosworth BOS as he's known the chief
technology officer of meta BOS great to
see you welcome to the show thanks for
having me yeah great to be here so I am
just off a demo of Orion your new
holographic augmented reality glasses
first of all they're crazy that's what
we're going for first and foremost
they're crazy I love that in Mark's uh
the super cut Mark showed they're kind
of Sizzle real it was just like people
saying like 10 people in a row just
saying oh this is crazy it's amazing
because you expect it to be an
experience you're having in VR goggles
I'll tell people you know just for our
listeners so these are glasses they
don't feel extremely heavy uh you wear
them just on your face like you would
normal glasses but they're augmented
reality so you can make calls on them
you can watch videos on them you can
play games on them and you have to like
remind yourself like oh wow like these
are these are glasses that's right I'm
not wearing goggles so you've had the
and this is a 10-year project within
meta but it looks like from what I saw
with your team that this is something
that has really come a long way even in
the past year that's right and you've
had an opportunity to use them what have
you been using them for yeah the the the
hardware came up last year at this time
and so the software bring up really only
came together a few months ago and they
just made incredible progress since then
already um the so for us there's a
couple different things I call out the
first one is actually very relatable I
suspect for those of us who worked
during the pandemic during the pandemic
you on these you know obviously doing a
lot of video calling and one positive
Silver Lining from all of that was the
the development of the back channel so
every you know meeting would have a chat
alongside the meeting and somebody could
ask a question hey I missed that is
there a link to that okay cool and so
you're not interupting the meeting to
get this extra context and what's been
fun with these glasses is these glasses
kind of serve as like the in meeting
back channel so now I'm in in an
inperson meeting in a live meeting and
somebody can add context to a
conversation that's happening um in the
glasses so I'm like oh okay somebody
references a prototype and then I the
next thing I know up Pops an image of
the Prototype that they were referencing
so now I'm fully in Conta again in the
meetings so these things are kind of
wild like these were before this it
would have been very rude to me in the
middle of this meeting with Mark
Zuckerberg take my phone out like go
look up the the reference and the thing
so that's been kind of interesting this
real life back Channel when you're in a
live scenario but let's also not
overthink it it's also a great way to
scroll Instagram and like enjoy the
reals and tap on those things um and to
watch videos the surprise hit I would
say from the demos has been video
calling um and there's two parts to that
right one of which is the experience of
having someone video call you while
you're just walking around is really
cool um and you can see them in full
Fidelity but you can also see and
navigate the world really effectively
you got your hands free and then
conversely what they would see is your
Kodak Avatar um and we've really fooled
some people with the Kodak avatars when
they've seen the Kodak Avatar call in uh
they don't realize it's Kodak Avatar
they think it's a real an image so let's
just go right away to like what this
does to Real World experience you you
mentioned it right so like it's so hard
to be in the moment now right and a lot
of times you're speaking with someone oh
there's a bz on their phone hey one
second let me check this out and then we
went a lot of people went to smart
watches I see you're not wearing one not
currently no and that also like you're
like sitting at a table with someone and
they're like keep on looking at their
their watch so does this take people out
of the real world in a way that you know
might be detrimental I mean what do you
think about the real world implications
here no tremendously important question
and we we've tried to use a tagline more
present and connected uh the idea of
kind of reducing the dichotomy we often
feel um and actually one of my anti-
patterns that I've been telling people
about for these glasses is just having
to be notification surfaces because
notification services are an invitation
to leave the present moment and instead
go elsewhere like that's what a
notification is is intended to do to let
you know hey if you want to you could
leave this physical plane and go dive
into this digital surface over there I
want people to be able to do that if
that's what they want to do but it can't
just be that um instead though if you
think of the examples I've given it's
about adding context and so the real
Difference Maker between these and a
phone in your pocket and all the things
that are attached to a phone in your
pocket is they have the ability to be
aware of the world at all times and this
is where AI has been such a tremendous
Tailwind for our vision for these um you
know we really thought early on we were
developing these even until maybe a year
ago it was going to be Holograms and
then at some point later Ai and in fact
it's going to be the opposite the AI is
going to come first it's already come to
to The Meta Rayband glasses um and so
for me at least we've got this idea
where okay what if these glasses are
contextually aware they know I'm talking
to someone face to face hey I don't need
that notification about my cat is moving
on the camera at home like I don't need
that notification right now we can hold
off on that oh someone's calling me it's
my kids school okay even though you're
talking to someone's face like we can
probably pipe that through and let you
know hey your kid school is calling um
and so that is the real missing piece
for allowing these to both be more
present and more connected when you're
using them is the contextual awareness
that they have the potential to have
about the world around you and how
you're engaging in it and there's also
just like a cool part of it that I saw
in the demo is um imagine you're just
like at a restaurant waiting for
somebody instead of like sitting and
scrolling your phone you could basically
have a computer with you which you just
brought in with your glasses you can do
some work you can watch a video do
whatever you want and it's there with
you yeah one of the fun demos that we um
we have in prototyping it's not even in
the demo that you tried yet um is this
cool thing where you you have your whole
workstation set up so you got like a
three panel workstation set up and then
you like just get up and walk around and
it goes away and it minimizes and now
you've got like you're walking around UI
if you need it um and then if you sit
down again you can just recall it
immediately so like all your context
comes immediately back whenever you want
it there so not just a a mobile and
portable workstation but one that's
literally you know at the you know
twitch of a finger uh ready for you to
use but look a lot of computing it
solves a problem right and another
question I have for you is why why do we
need this like what type of problem does
this solve can't we just get along fine
with our phones and I know that's going
to sound stupid because every Computing
phase they've been like well can't we
just use desktops why do we need this in
our pocket but so what what problem do
you think this solves no it is it's a
it's exactly the right question it it
speaks to how we as a society so quickly
normalize everything that's come before
as completely normal and everything
that's new is like kind of this
unnecessary thing Douglas Adams famous
quote on this uh comes to mind um I'll
leave that as an exercise to the to the
listener the so for me at least that you
know when people I remember when cell
phones came out and everyone was like
who is so important that they need a
phone everywhere they go and it's like
all of us are it turns out all of us are
that important um and I feel a little
bit the same way and some of these
things will sound trivial but uh having
meta AI is actually a really good test
Point having I've had meta AI on my
phone for a while as an employee of the
company um and it's pretty useful and I
like asking it questions um now I have
meta AI on the Rayband metag glasses and
it's like oh I use it 10 times more
often uh because just the friction of
like getting my phone out of my pocket
and like pulling it up and like pulling
the app is not trivial to me be able to
just like ask a question um and now what
if instead that AI didn't need to even
be asked a question it's anticipating it
sees me I'm like checking my pockets I'm
checking it's like oh your keys are over
in the desk where you left them um it so
these things start to come together
where because the glasses are on your
face they're not in your pocket and they
have the ability over time not
immediately to become aware of the world
and to understand what you're trying to
do in the world uh they can actually be
very proactively helpful just to you
privately you're in control um but that
is a thing that phones can't do uh and I
do think it is a real step forward in
our relationship with digital
Technologies from going very
transactional like I have gone to the
technology and then I come back with my
answer to being very integrated so is
this basically a way to just deliver the
AI platform more seamlessly it's not
just the AI platform I think the AI
platform is what changes the scope of
how we interact with it um from being
completely reactive which is what
computers have been from time immemorial
to being proactive um I think it also of
course is a super more convenient way to
go ahead and manage communication manage
like manage all the different things
that phones do listen use video calling
again as an example you know this is
just a terrible way I'm holding my phone
out my hand out in front of my face like
my arm is literally tired I've been
doing this for eight seconds and I'm
like I work out man like it's like I
can't so there's a few things that are
just fundamentally better on this
platform um and then there's a few
things that I think yeah you can do it
on the phone or you can do it here but
if this was already on your face you
wouldn't use your phone right I mean
I've definitely had those moments where
I'm like video calling in New York and
just running into people absolutely I've
walked I've walked into a post before
like I'm not proud yeah um so why does
why is it important for meta to do this
so meta's Mission long has been all
right let's you know connect people
obviously it was the name change now
it's metaverse or you know meta um why
is it important for this company to
build this device yeah the there's two
different answers to this one that I
think people will recognize and and
understand strategically we have just
been at the mercy of platforms for um
since the start of the mobile era um and
we were a web company right we we were
born of the web and the web was this
like amazing unowned platform that you
just like um really a lot of control
over what you built and get a direct
relationship with consumers and we
absolutely feel that in our application
development we're stifled by The
Gatekeepers especially Apple in terms of
the the policies what they allow what
they don't allow and there's real value
that we could have delivered to humans
who love our products and would would
have be happy to have that value but we
weren't allowed to we're just not
allowed to um well anything that has to
do with you know story keep in mind on
the web Facebook was a had a major games
platform um you know which we were not
allowed to bring to
thees um and gosh I mean and they've
gone even further than that and kind of
you know F we built a app install
business and then Apple came for that
with ATT you know a tremendous amount of
of kind of self- preferencing for
themselves there and in doing so they
really put a you know the breaks on what
was a really rapidly growing part of the
games industry um and so anyway so I
think like for us like platform control
is important to us that's a strategic
reason that's what everyone jumps on
that's a fair one if I can be honest
with you that's like a very nice story
and it's I'm sure it's a part of it but
it's not the real story the real story
is that we're a technology company and
Mark is an Enthusiast who wants to
continue to advance what is the
state-of-the-art and how we connect
people with each other and with
technology and we've known for a long
time that these are the next thing as an
industry I think we've had a sense AR
especially is like we don't have to
convince people that someday AR is
probably a thing that will exist and
will be great um and this is where
Michael abrash our chief scientist has
this kind of stat like you know this is
the myth of technological eventualis
this idea that eventually that
technology will exist he like it's it's
doesn't that's not how it works someone
has to stop sit down and actually build
it um and it's a tremendous credit to
Mark I'm G to sound like a kiss ass I
don't mean to I'm very critical of Mark
in many many areas but on this he
deserves his credit it's a tremendous
vision and commitment that he's made to
this technology in the face of
incredible scrutiny the last several
years to stay with these Investments and
deliver Orion which I think is just a
watershed moment for us to say like this
is this technology is coming it's here
um in some form it's not evenly
distributed um but it's here and and we
it's like within it's within our
lifetime certainly within the next
decade it's in our view we're going to
be able to to use and enjoy this
technology um and I think he just wanted
to do it and we should say that these
glasses are internal so over time they
might be released they're not released
yet that's right but I do think we
should talk a little bit more about the
platform side so okay for listeners I'm
sorry just put on I put on a pair of of
uh clear and transparent ones the the
ones that we actually use are uh have
opaque magnesium shells the clear
transpar ones are super fun to have uh
they don't they only work for about half
an hour at a time before the thermal
shut down because this plastic does not
dissipate heat fast enough U but real
the real ones uh they don't get they
don't thermally exhaust themselves they
go till the battery is exhausted two to
three hours um but yeah so it's pretty
fun it's just I'm sorry if I have these
cool demo props what fun I can't put
them on but let's talk a little bit
about the the platform risk and and the
operating system mean do you have to
build your own operating system for
these yeah so a couple different stories
to this I mean when we talk about what
is an operating system uh one of my
former colleagues Fus kpatrick had this
great saying like operating systems are
a set of promises made to developers and
to Consumers so you can think of um in
uh iOS um the swipe gesture will get you
home like no matter what's happening on
the app that's a promise that you're
making to a consumer and the apps can't
control it that's a promise that's being
made um you know you promise to
developers like this memory is yours
you're allowed to use this memory this
API will cause this thing to happen um
and so you know we're we're all we're
using Android at the most underlying low
level um as as the operating system but
of course the operating system isn't
Android the thing that people are
interacting with on whether it be uh in
mixed reality or in augmented reality it
doesn't look or feel like Android and
there's a bunch of reasons for that
number one is the interaction paradigms
are different um and so for augmented
reality um to stay there for the moment
you're not going to have a direct
manipulation interface you don't have
keyboard you don't have a mouse you
don't have a touchcreen well it's worth
noting this is really the first
Computing platform that we've had ever
that didn't have a direct manipulation
interface I mean you're going back to
Stanford Research Institute Xerox Park
the keyboard and mouse um before that
was he was all main frames the keyboard
and mouse then you had this tremendous
innovation in touch screens and multi-
touch screens direct manipulation very
akin to the mouse every single
application you've ever used you've used
via a combination of direct manipulation
we don't have any of that because you're
not going to walk around with those
things you're not going to have touch
controllers with you as you go about
your
so what are we using we're using hands
which can sometimes be modeled into
direct manipulation with other little
laser pointers like we do in mixed
reality U but ideally you're using uh
voice which is not always appropriate
and ey tracking an EMG which is again
approximating what you had with direct
manipulation with your hands and so for
me at least like when you change the
interaction Paradigm that is a half of
what an operating system is it's like
how do you the consumer affect the
machine the other half is how does the
machine bring information back to to you
and here we're more similar you've got a
display you've got audio except for the
fact that the display is additive light
so there's light coming through from the
world around you so the display has to
be somewhat aware of what is happening
in the world around you ideally um to be
effective and so anytime you're building
a new uh interaction Paradigm um
especially with when pairing with novel
display technology you're basically
building a new platform um there's not
really another way to do it and so then
how do you think about like who might
who else might build this you were just
speaking with Ben son you talked about I
don't know if you were saying the
potential for Apple to build something
like this but you talk about I'm just
going to quote from you you look at what
are Orion glasses they're full AR
glasses you have custom silicon in the
glasses custom silicon in the puck but
Apple could build all this and just be
like oh it only works with us so does it
fully insulate your yourself from the
platform risk no nothing fully insulates
us especially in a world where mobile is
the dominant platform expect them to
build something like this as well I have
no idea uh you know they they keep it
pretty tight uh down in Cupertino they
don't they're they're a pretty tight
shift and so I don't have great
information on that um I think they're I
think they would love to build something
like this I think we've been investing
longer I think we've believed in it more
um and as a consequence of frankly Mark
Zuckerberg's Faith um and tremendous
execution by a team of thousands of
people over a long period of years I
think we have a meaningful lead in a
bunch of the Technologies here uh but
you'd be foolish to count out a great
technology company one of the great
all-time companies in general uh at
Apple um but my point is like I think
that the fact that they already have
this phone that's in your pocket has
some advantages because you now got this
device that has a bunch of compute it
has a bunch of silicon they can do great
custom silicon they can add more to that
it's also a disadvantage I think in a
way are they going to be willing to walk
away from you know one tremendously
outsized profit Center in pursuit of
something that is fundamentally
disruptive to that profit Center um and
if you're centering more of the
experiences on the phone as opposed to
on the glasses do you make the leap that
you need to make in developer platform
in interaction design um because a lot
of those things that we've discovered if
you really were to Center them on just
the phone and the way phone apps want to
work they don't work as well um so I
think there's like it's it's it's going
to be a dynamic uh it's going to be a
dynamic landscape for a while um and I
think we take some risk sharing this
with the world early to demonstrate
what's possible but we do want to ignite
The Passion of the developer Community
the engineering community around us um
and I think it's also something that we
kind of owe to people people so they can
see what this 10e journey we've been on
in reality Labs is all about okay so
let's talk a little bit about the um
reason to have an app it's just the
normal way that you use a phone um but
you're talking about these glasses
they're sensing the world you know you
now have uh AI voice that you can speak
with I think Mark yesterday said or
we're talking a week before this is
coming out but he said I think that
voice is going to be a more natural way
of interacting with AI than text which
is basically interacting with Computing
if you think think for sure that's the
way this is all moving do we need apps
anymore it's a great question I love
this and actually it goes back you know
applications didn't always exist right
in the beginning it was just like the
system was the software it was running
at the moment and you would kind of
program it um and I think often about um
jcr lick lighter sitting down for the
first time at a terminal which is
reprogramming itself in the fly it's
really one of the first times um in
history that people are doing human in
thee Loop Computing as opposed to what
would have before that been through
mainframes and time shares you put your
job in you come back later get the
results of your job if it was wrong you
change your software and try again this
was humanin thee Loop Computing and it
was this tremendous watershed moment
that ultimately LED through you know his
work at DARPA the ipto uh with Doug
engelbart with Xerox Park to the alto
and that's where you start to see oh
there's like a windowing SRI like
there's a windowing model and each
application is contained and the
application was born um as a consequence
of us being in Human loop at the
Computing um and what's crazy to me is
the analog that I draw to chat GPT in
fact which was this human in the loop AI
whereas before this you would like
create an AI and it would go do a job
and come back and if the job wasn't
right you'd like reprogram your AI
whereas this is like no no like I can
just modify my queries on the Fly and
the AI is like adapting to it um and so
one does Wonder like um does this get
into a more if not an app free world a
more webik world again where things are
deeply linked and interconnected through
services to one another um I think it's
one of the challenges that the app model
is going to face like things like apple
intelligence for example will face on
the phone is what I really want is for
something that actually intermediates
the apps and can go to my Amazon app and
grab the my purchase history and then go
to you know a different app and drop
that in there and add that to my
spreadsheet for my budget which is in
this third app that I have and that is
like that is not something that Apple
wants some other developer to be able to
do because that would disintermediate
their position their privileged position
on the phone
um and so like if the AI can't work
across all the breadth of my
applications well it's not the most
helpful AI it could be one thing that's
really fun about the desktop and the web
still is that it can still work that way
um and I love those platforms for that
um including OSX um and so I think often
about okay if I once I have this AI I
want it to work at basically user level
like the AI is actually my agent in the
machine uh I think you probably end up
still having applications there's a lot
of really valuable constructs we've
built up as an industry over time that
have to do with the Safety and Security
of data and the model of how these
things interact with both the computer
itself with the network and with other
users so I think there's actually still
quite a bit of value in having these and
not to mention just how literally would
we do it and have these systems not
constantly walk on each other so I think
the application model doesn't go away
but I do think you end up injecting this
potential new layer on top of it which
is an agentic layer um and so wait how
how do developers then think about
building for this type of Technology it
becomes much more about what apis you're
providing and the unique value that you
provide through your the software below
AP it's about the API you're providing
does that mean the main interaction
layer is going to be through some sort
of voice AI or it's hard to say at this
point I think that's voice is one ver I
don't I think voice has a great modality
for some things it also has a lot of
vagueness to it even amongst humans a
lot of context that's required even
among humans that obviously even a a
really great agent wouldn't have for a
long time if it was with you um and so I
think voice is probably not the only
piece of it Voice and text are part of
it I'm sure there will still be a lot of
very direct manipulation interfaces for
a long time to come um but you can start
to imagine yes as voice gets more
developed my ability as a developer to
create value in how I manage process
data or how I you know um manufacture
new artifacts is actually the more
important thing I'm doing rather than
having to also extend that to the
consumer through some kind of interface
so in this world do the AR glasses
replace the phone or do they supersede
it in some way um I can't imagine it
goes
away we there it's a good question we
think we think about this a lot I don't
it doesn't go away for a long time if it
goes away at all there is a real strong
argument that um when the AI glasses
become super prevalent and that is your
interface to a device um and you have
especially if you have a display whether
it be on AR glasses or it be on your
phone or your watch or whatever the
thing is that you can kind of when you
need to directly manipulate something or
provide very precise entry you can um
you don't need a phone anymore like it
becomes it becomes what the phone is
today we're a long ways from that for a
bunch of reasons not the least of which
being the technolog is not ready yet but
also because the application ecosystems
will take a long time to transfer and
user behaviors take a long time to
change so I think we have a long
timeline there um but like I do think
it's conceivable that for a lot of at
least you know high-end capabilities um
and consumers that AR will replace the
phone so what does like Computing look
like in that world uh hopefully it's a
lot more fluid um hopefully it's a lot
more self-aware I mean Computing today
is like it's hilariously dumb actually
like it's like we all like computers are
amazing at what they can do and we're
impressed because like they can do math
really fast um but like if I like type
in my phone that I want to watch
instagrab it like it's like sorry
there's no app called instagrab it's
like I meant Instagram it's like I
didn't mean instagra it's just a typo
but like the machine's like no I'm sorry
you clearly wrote instagr here and I
therefore will not get you the app that
you want um like computers are dumb in
that way um they have no construct of
what my intentionality is children human
children can Intuit it the
intentionality of adults like very near
infancy it's like incred like um you
know I was just talking to someone
recently who told me about these studies
that they do with um human uh toddlers
or actually I think even maybe infants
who are watching their mother interact
with somebody and if the interaction is
very kind they start to build a model of
the person the mother's interacting with
and if the interaction looks
transactional then they just ignore the
other person exists at all like there's
like early on like we're like okay
there's some intentionality in the world
that's happening theory of Mind of other
people that exist and like what are they
trying to accomplish and what am I
trying to accomplish computers have none
of that and it's the fundamental way
that we as people interact with one
another um we are we understand what
each other is trying to accomplish and
we then react accordingly um and so I
think that's the thing that the thing to
look forward to the most about the
future of computing is if you have a
device that is
instead of being throttled on the
bandwidth of how much information you
can give it is actually throttled in the
opposite way it has total context on you
uh what you've been doing what you're
trying to do right now what's possible
what's what tools are available to you
um and then has enough um intelligence
to do something useful to that to give
you either visually through a display or
audibly to your ears or through haptics
the hints that you need to then just go
be successful what you're trying to
accomplish it starts to feel instead of
hey I'm a human with a great tool it
starts to feel like superpowers it
starts it feels like oh like I just have
tremendous more capability than I used
to okay I have some follow-ups to that
but I'm going to ask them other side on
the other side of this break where I'm
going to get a little weird so we'll be
back right after this and we're back on
big technology podcast here with BOS
Andrew Bosworth the chief technology
officer of meta BOS great to see you um
so I promis some weirdness before the
break and uh let's do it uh is this you
talked about like uh humans and
Computing getting closer and being more
context aware of each other is this AR
move just a step toward where we
eventually go which is that we just
eventually insert Computing into our
brains through brain computer interfaces
it's a great question and and you
already have the neural control for the
device that's right the neural interface
on the wrist um which the you know the
is already very successful and we think
that's just the beginning think we can
do a lot more with that over time right
and for listeners so like you have this
wristband around your wrist and you're
pinching and it selects you tap your
middle finger goes home you can scroll
by just like putting your your thumb and
pushing it on your hand like it's nuts
sorry go ahead and it's just the
beginning over time we think we can do
that without having any gestures at all
just think it just having the intention
expressing the intention um down your
your control system so a couple things
you know eight years ago we did an
invasive um neural implant with in
partnership with UCSF it was a patient
who was non-verbal due to a traumatic
brain injury and we were able to provide
them a 400w lexicon by virtue of this
these implant that we gave them um and
um and the reason we did that study was
then when the when the person was using
the the the control system we put super
sensitive uh EMG sensors on their head
to see if we could detect any um pattern
of the underlying neural activity
without having to be invasive uh we
couldn't it was actually your your your
skull is an incredible insulator of
electrical activity and it's kind of
scrambled together um the challenge with
neural implants um we've actually been
doing this for even longer than that um
you know the first people to be able to
control a computer M through mous
through a neural implant goes back um
into the 20 early 20110 if not earlier
than that um there's a couple challenges
one of which is that the brain melinates
the things that are put there so they
become electrically inured over time um
so they have to be uh removed um and
replaced uh and if you leave them in
long enough they can prove fatal or at
least have very severe comp
complications so you wouldn't just be
talking about with the technology that
we have today with the materials we have
today and what we know about how the
brain works um you're not just talking
about a one-time brain surgery you're
talking about annual brain surgery and
annual brain surgery for me is like a
non-starter as a consumer product I'm
just going to go ahead and assume that's
not going to be the way things are going
to go um and so I guess I'd rather wear
the glasses than that yeah right I think
I think the glasses are a popular option
uh in in the relative to that and so we
um we're tracking that space carefully
certainly and thrilled to see people uh
investing a lot to try to see if they
can crack those problems but one of the
things that we keep finding is you don't
necessarily need to go to that point um
and the glasses are pretty spectacular
and like it turns out once you get to
all day wearability with their their
lighter and they have full battery life
the ability to take them off is actually
pretty nice the ability to just like
have you know to kind of be restored to
that Center um does it get to implant at
some point yeah maybe I think that's
probably probably light I don't feel
happens in my lifetime really um it's
like that feels like um even distant by
by my standards I think we like have in
and maybe AI is a key to unlocking this
more I mean I studied one of the things
I studied as an undergraduate was a lot
of neurobiology was kind of part of my
interest in in Ai and computation um and
like the the challenges that we have
understanding the brain uh feel like an
order of magnitude bigger in terms of
just complexity than anything we've
managed to actually map out so far um
you know so this idea that we're going
to merge with AI you're not buying it no
I don't see that happening in the near
future no I I I'm not saying the would
you do it if you
could we are getting weird I appreciate
you good is delivered I promised um I
think what I think I really like um
there's a a book uh Hof's book I am a
strange Loop um which has this idea of
Consciousness as an emergent property of
sufficiently self-referential systems um
and it's a beautiful book and uh
rumination unconsciousness from you know
one of the best to ever do it and it
also is kind of a beautiful pan to his
wife his late wife um and he has this
idea that um his wife who passed away
unexpectedly there's like a version of
her Consciousness running in his brain
right like that's been uploaded to his
brain because he can imagine with High
Fidelity and this gets back to what I
said earlier about you know the idea of
understanding someone else's
intentionality and and agency he has
like a pretty reasonable approximation
of her Consciousness in his
Consciousness it's true like if you if
you're close enough with someone you you
can effectively like speak with them if
they're not with you absolutely uh and
we have a mod a version of their
conscious running to it um so I I think
if you got to the point where an AI was
everpresent in your life um and helping
you guiding you you're confiding in it
um maybe even your innermost thoughts
that you wouldn't normally even share
with a close friend or someone else um
that it would have in its systems a
version of your Consciousness aqu
queriable like interactable animated
version of your Consciousness is it you
that's a I'll leave that to the
philosophers um but it would it would
passes you in many contexts and and
obviously there's a lot of questions
that raises in terms of privacy and
security and how you control these
things but assuming we can conquer all
those things yeah I'd be thrilled I'd be
thrilled for my kids if something were
to happen to me to have access to even
if it's a Similac um a a construct that
approximated
what I was um for their use and I'm sure
there's going to be a whole new set of
challenges that that introduces um but I
think that's a a version of the future
that is likely do you think that the
current systems are conscious or
approximate Consciousness no I don't
they're not getting close to it no I
don't think so I think it's a category
error to be honest with you and I'm a Hu
I find this that's a funny thing that
happens me these AI conversations
because I am both one of the most
fanatic Believers in the power of large
language models I talked about how I
think it I think of it in the
fundamental as a fundamental Computing
Paradigm Shift of going from um human in
the loop Computing to human in the loop
AI where the AI is much more informed by
modern sensors and availability of
information um so I a tremendous
believer in it all and also I do not
believe it is even the kind of thing
from which Consciousness exists um it
doesn't exist in any again to my point
about Doug hofstadter um it doesn't
exist in any self-referential Loop in
fact it doesn't exist at all except when
I instantiate it to exist it's a very
very cool associative database um it
reminds me actually when I was an
undergraduate the reason I studied this
computational neurobiology certificate
that I got was this idea of what could
how can I with a computer emulate human
associative databases human memory is
this wonderful associative tool where
when I say the word purple to you um
this huge network of of ideas pops into
your brain um and it's like very rich
and complicated and it's very personal
to you and what your history of that of
with that language is um and it's a
really hard problem for computers to do
that like it's to to have that kind of
loose and generative associations um and
so I can say to AI today I can say to it
like um imagine a purple elephant and it
can do it and that was kind of wild
before there was no way to do it before
so I love it but I do see it as
fundamentally an associative database
and an important maybe a component of a
total system but I tend to think Yan
laon's uh critiques of it are spot on
are spot on okay so let's rapid fire
through the rest of my questions we have
a few minutes left Let's Do It um what
happens to VR or these mixed reality
devices the quest and the rayb band
metas if Orion takes off the analogy I
make is to your phone and your laptop I
mean the truth is there's a and this is
what we found so much and the reason
they have different operating systems
though we have tons of shared technology
layers and sensors and hand tracking and
eye tracking and uh voice and AI shared
technology layers different oper is
because the interaction design is
different and the reason that's
different is because the use cases are
different just like um I tell the story
all the time internally but most people
who are in information work understand
it sometimes I'm on my phone I'm
responding to an email and my phone does
email fine and I'm like not a phone
email I have put my phone down and get
my laptop this is a laptop email I need
to have five browser windows open I need
to have like I need to this is a real I
got to write this thing and so I do
think of these things as like
fundamentally different pieces I I
certainly watch quite a bit of short
form video on my phone I also get the
big screen at home with my wife and kids
and watch a movie and it's just like
they're different things and I can't
describe to you in human terms why
they're different but you and I both
know that they are different and they
feel different and so I think there
continue to be even when you have these
wonderful AR glasses um things that can
be done in Virtual and mixed reality
that can be done no other way your
augmented reality glasses will never
convince you that gravity is off um
because you can literally see the
physical world behind them when you're
in the the headset you can absolutely
play as if gravity doesn't exist and so
I think there are they both serve
tremendous value and I think of it as as
being a phone and a laptop again
speaking with developers um they have
lots of options build on the phone build
for these devices that you know are
starting to get a uh you know user base
but aren't anywhere near as big it's
tough to reach scale um what's what's
the one minute uh pitch to them if
they're thinking about building for
these devices these VR mixed reality or
ar devices um the mar AI voice for sure
the Market's super efficient and so y
there's a lot of people on the phone but
it's also the hardest place to get
noticed or stand out it's the hardest
place to do new things because there's
nothing new that you can do there it's
all been done before phones haven't
changed in 10 years um whereas these new
devices are uh risky because the
audiences are smaller but you have an
enthusiastic audience who's deeply
engaged and is absolutely desperate to
see new things to try new experiences so
if you have a great idea um you know it
feels like a gold mine uh it feel and
walking around connect the last couple
days has been so energizing because well
I'm surrounded by people who have
thought of ideas that we never
considered when we built these platforms
and they're just powering uh powering
ahead and pioneering those things so um
I can't listen every developer should
should kind of probably play to their
strengths and and understand there's
these advice these devices and AI in
particular all live in very different
contexts um uh but it's going to be hard
to break out in the phone um and it's
designed to make it hard for you yeah
because I mean like without the
applications from from developers it's
going to be tough to find like Great
Value in these devices that's why every
time we talk about Quest we talk about
the ecosystem right it's not just the
devices I'm not here to sell devices I'm
here to build an ecosystem and that's
about building getting more developers
to build great experience to get more
consumers and more consumers which make
it attractive for more developers okay
couple AI questions before we end uh we
have a debate often on the show is it
going to be the development of the
foundational models or is it going to be
the application of them where the real
value is found in artificial
intelligence obviously you guys are
open- sourcing yeah the foundational
model hand on that answer you have an
answer so talk a little bit about do you
see promising applications being built
and um what happens to the big
foundational model companies if this
stuff is all effectively commoditized
yeah so I was one of the loudest voices
internally um to CH encouraging us to
open source llama uh one llama 2
obviously you see how boted Mark is with
his announcements around llama 3 um and
the reason I for that for me is like
it's a tremendous benefit to our company
whether we make the AI or someone else
makes the AI good I'm the only one who
can use it in our products and that's we
have a direct relationship with our
consumer so we can make our products
better for consumers whether we make the
AI or someone else makes the AI so that
makes this um a a compliment to us and
you want to always commoditize your
compliments um you always want to make
sure that the thing that you depend on
is like cheap and getting better by a
lot of people not just by your effort
and money um and so strategically it
makes perfect sense but more than that
we're a technology company and we've
always seen AI in particular as um the
the data we bring to it for our own
applications that's special to us and
the application itself is special to us
but we've always with pytorch we've
always tried to make sure that the
middle piece that the the the
infrastructural pieces those are for
everybody um and that like they they
that makes our developers have better
apps um when they're on building for
Quest it makes sure that the partners
that we're working with um you know for
Instagram are having better experiences
it's really good for us to have the
entire consumer base whether they're on
our app or somebody else's have a better
experience um I think for us though to
be honest with you it's hard to say what
happens to these companies I would have
been wrong about AWS if you'd asked me
this question a long time ago because we
were building the lamp stack we were
doing it all ourselves and it was
everyone was doing it it was fine and
then it turns out if you can abstract
away at good enough economics the
complexity of managing a bunch of web
servers and load load balance and
scaling up and scaling down there's a
business there so I think there probably
will be a business there for some of
these AI hyperscalers it's probably you
know potentially a big aw is a big
business but they've got to get to that
point where the economics make sense and
it's sufficiently well abstracted from
from them I think the bulk of the value
though so there will be valuable
companies there but the bulk of the
value will be in the application side so
I was driving down 101 today which I've
been doing for I was thinking about it 9
years coming down to visit you guys it's
an interesting moment right we are ahead
of November 2024 we're not talking about
politics um I was also like thinking
about the different types of technology
that I saw when I came down here and I
mean you know credit to your teams but
it was like stickers in messenger and
360 video sure what is this technology
moment look like to you because it I
mean seriously we're talking about
artificial intelligence that can
effectively get to know us glasses I'm
looking through earlier glasses that you
can like see through that's right and
perfectly clear those are clear that's
right no I'm like thinking about the
lens yeah totally just like uh uh it
looks like a VR device pretty much but I
mean from the inside so talk a little
bit about this moment and do you think
we're going to continue the momentum or
the tech industry really will continue
the momentum because it seems like
progress is nuts right now one of the
things that we talk about all the time
as technologists is these overlapping
s-curves uh where a new technology is
created and then has this period where
it struggles and then it gains adoption
and then it explodes and it's everywhere
and then there's still a tremendous
amount of value at the top but it starts
to like the rate of change starts to
slow down um and so you know I'll take
an easy one you talked about messenger
stickers and it's easy to make fun of
messenger stickers in retrospect except
they're still super popular and we
wouldn't trivialize emojis emojis this
idea of expressing yourself through
Visual form as opposed to expressing
yourself explicitly through text um was
at first a laughable construct that like
our generation got made fun of a lot uh
by the older generations and then now is
like ubiquitous among all generations
and arguably helps us um express
emotions through a medium that otherwise
lacks the ability to do so so cleanly um
and so I don't like to trivialize any of
the things that have come before but I
do think it's fair to recognize that 10
years ago we all felt like we were at
the top of an scurve of of really what
was the kind of tail end the web we
writing the mobile curve straight up the
ramp at that point so the web is kind of
starting to Trail off mobile is like
still going gang busters um and it's a
tremendous you know foresight for Mark
to think what's the one behind that mhm
like what's what's the one it was crazy
when I was here I was trying out your AI
assistant and messenger called M that
was actually contractors in the
what'sapp building that were
approximating what it would look like
and we were all getting this wrong as an
industry two years ago a year ago you
know I remember when we had uh portal
still one of our most beloved devices
and I'm sorry to all of those who for
whom uh we had to shut that down yeah
look into the camera until Katie npis
Katie Katie you know how much I
apologize it was and and and our my no
one more than my our own employees our
own employees were I devastated when we
to that tremendously popular product
small Market but tremendously popular
product and we had our assistant on that
and we had an Alexa assistant as well
and when we got into building our own
assistant I could not believe the how
poorly they scale because you have to
build it for every language for every
device and for every use case oh my gosh
that's not software that's that's that's
the worst scaling laws of all time and
so even back then I knew this isn't it
like we didn't do it and it wasn't until
you get self-supervised learning
combined with um the breakthroughs that
that Google Led in um in how they they
built these um these models and then
ultimately chat GPT that you see it
you're like Ah that's what it is like
that's the thing that's the one that we
needed all along and we didn't have it
yet um you know what they say in the
industry though there's no bad ideas
just bad timing um and M was was
probably one of those okay Andrew
Bosworth great to see you thanks for
having me in thanks man thanks everybody
for listening and we'll see you next
time on big technology podcast