Amazon Reveals Its AI Master Plan — With Matt Wood

Channel: Alex Kantrowitz

Published at: 2023-08-03

YouTube video id: 2eCtvzVHuCk

Source: https://www.youtube.com/watch?v=2eCtvzVHuCk

good so Matt I I did my research about
where Amazon fits in the generative AI
space
and I looked at like the chat CPT model
and I looked at chips okay you're
dabbling that in that area in those
areas but I don't know if your standout
there yet but where you are really
trying to compete is in this space where
companies the big companies bring their
models inside AWS and then anybody that
wants to build something with an llm can
build it through your products so talk
to us a little bit about that initiative
and why did Amazon pick that in
particular
uh sure uh number one I would say we're
doing a lot more than dabbling uh I
think we've got a very meaningful uh
focus and investment just across the
company on generative AI
um myself I think the rest of the team I
think like a lot of other people here
probably the light bulb came on when we
started you know playing with chat GPT
when that first came out
um we really got excited and inspired by
the capability here and so what we're
trying to do is maybe a little different
from what some other folks are doing
what we want to do is take this
technology and make it as broadly
available as possible there's a lot of
these kind of magical interesting
Technologies like cloud computing in 20
years ago like machine learning 10 years
ago and now artificial intelligence that
have traditionally been only available
to the very very largest technology
companies to the biggest governments and
academic agencies and so uh my mission
and our approach is that we want to make
that as as broadly distributed as
possible we want every Builder and
everyone to have access to the same
capabilities that were once you know
very very limited and so I think let me
challenge you on that right off the bat
I mean there's a huge open source
movement in this area in fact Facebook
just released this llama 2 model open
source you can use it customize it you
don't have to pay them a thing so there
is access so where is that Gap that
you're seeing between you know the the
high barrier and what what is available
on the market absolutely so uh Lama two
is a excellent very capable model but
there is a long way to go from having
the model weights which uh what
comprises the neural network to actually
building out an artificial intelligence
system and just having the model weights
is super useful but it's like it's like
having the source code to some software
yes it will give you some capability but
you still need to be able to deploy that
somewhere you still need to be able to
understand it enough to be able to make
changes to it you need tooling that
understands how it works so you can
actually take it as an engine and put it
inside the car that you're building and
so what we're trying to do at AWS is
make it really really easy to take llama
2 and other models like it whether
they're open source or proprietary and
use it as an engine inside cars and
boats and planes and all sorts of things
so tell me a little bit about the
process so Facebook comes to you or meta
comes to you and says hey
Matt we have this cool open source model
we'd like to make it available to your
clients through AWS is that sort of how
the process goes I mean pretty much
um and what we do then is we take the
the model and the weights and we put it
inside a capability our machine learning
service that we have called sagemaker
and sagemaker lets any Builder build
train and deploy using machine learning
models and llama 2 is one of you know
several dozen large language models that
are available on stage maker today
that's exactly almost exactly how it
happened actually right so so sagemaker
is your managed service that allows
people to build to shape models but you
have a newer product that's called
Bedrock which allows people to build
things like agents for instance and they
get to it's very interesting they get to
pick the different model that's right
that they want so talk a little bit
about how how that works and and I'd
really love to hear you know am about
Amazon's progression from sagemaker to
bedrock and and why why that puts you in
a in a better strategic position sure I
think sagemaker we launched 2017 I think
and it's been very successful very happy
with the with the business customers
love it many of our customers have
standardized on sagemaker for their
machine learning workloads
um but one of the super interesting
things about generative AI is inherently
because you're not training the models
yourself you're taking models from
Amazon and meta and a whole host of
other stability Ai and just building on
top of them it makes machine learning
far more accessible and so while
sagemaker is great at building and
training and deploying those models we
wanted to find a way which gave the
maximum leverage of that accessibility
so you could just instead of having to
figure out how the model worked and
fine-tune your data and all those sorts
of things just give a prompt just tell
us what tell the system what you want
choose the model that you want to run it
against and we have about half a dozen
models plus include from partners and
some from from Amazon and then we just
give you the result there's no servers
there's no infrastructure to manage you
don't have to worry about using data or
labeling data or worrying about gpus or
capacity or any of those things just
give us a prompt put in choose the model
and we'll give you the output so it's
pretty cool like if you wanted to make a
chat bot for instance like people think
all right I want to make a bot I go to
open Ai and I build it with them but
what you could do actually in Amazon's
technology is go ahead and build an
agent or a bot and then pick whether you
want
open AI or Claude right it just it
depends it runs the gamut that that's
the Strategic bet for Amazon that's
right and then our approach is to find
areas that are really really valuable
the customers real problems the
customers are trying to solve and then
add capabilities to bedrock to make
those problems smaller so a good example
would be a chat bot so chat Bots like
you may have played with chat GPT
um
they're very capable they can understand
what you're talking about they have
context you can go back and forth and uh
and they give the appearance of
intelligence but they actually are not
very good today at completing complex
tasks and so let's say you wanted to
create a right retirement plan you could
ask your chat bot build me a retirement
plan and it would go off and it would
build a very kind of
reasonable approach to retirement that
can't pet my retirement officer I would
I would vet that very carefully but it
will give you a pretty good strategy a
pretty good starting point but it
doesn't know about your personal
finances it doesn't know about the state
of the markets it doesn't know what uh
investment products are available and so
one of the things we're adding to
bedrock is the ability to be able to
provide that information to the language
model using your own private data inside
the applications that are already
running the Amazon Cloud and to be able
to extend the language model's
capability with that data in a couple
minutes so that the model can produce
not just a strategy but it can actually
help you complete a task and that's
something that hasn't been possible
before you know what Matt this this all
sounds good and then I go back to the
release of Lambda 2 last well a couple
weeks ago
and okay I saw it was definitely on AWS
but Azure is the preferred partner there
so
yes you're doing this it's a it's a
strategy that makes sense in my mind but
you're also your competitors are doing
it as well so
where's the distinction there well I
think the distinction is that we have a
slightly different approach to some
other providers we want to broadly
democratize this technology we want to
do that because we think that there's
not going to be a single model to rule
them all and so others are talking about
well our our stated goal is that we want
an artificially generally intelligent
system that is not our stated goal our
stated goal is that we want to just be
very pragmatic meet customers where
they're at today and then provide
capabilities like agents provide the
option of different models and then
allow customers to deploy those
capabilities in a way which is very low
cost High availability and low latency
and that operational performance is I
think something that's going to really
set folks apart in the future okay but I
have to get back to this Azure example I
mean they're doing the same thing so are
you and Microsoft are you and your
you're not going to have I think what
you're basically saying is you're not
going to have this field to yourself you
realize that you're going to come up
against competitors doing the same thing
or maybe I'm putting words in your mouth
well I think uh I think it's safe to say
this is going to be a very competitive
space for a very very long time the
opportunity is enormous the capabilities
are incredibly early and so when you
match
early capabilities with a huge
opportunity you're naturally going to
get a ton of different competition and
ideas and thoughts and we have our own
and who knows if they'll turn out to be
right or not but our key point of
differentiation is to be able to allow
Builders to be able to build these
systems with their own data privately
and securely and to be able to Leverage
The Investments that they've already
made in that data on AWS
using completely novel capabilities in
addition to existing models and novel
models as well so there's a lot of focus
on models today but those models are
going to remain important but over time
there's going to be additional
capabilities like agents like
reinforcement learning like the ability
to be able to understand and vet the
responses that come out of these things
are accuracy that are going to be as
important as the models over time one
more Microsoft question if I may sure
they're so deeply invested in open Ai
and those GPT models
how does that make
you distinct from them like are there if
okay put me in the in the um in the seat
of a customer who's evaluating these two
solutions
openai Microsoft got all the buzz in the
beginning
but Microsoft is totally sort of all in
on this model you have your own models
we're going to talk about them but
Amazon seems to be less it has less at
stake in terms of making yours work so
so
is there a point of differentiation
there if I'm a customer like for
instance you know is it more neutral
like how do I think about that how do
they think about that yeah I think um uh
we're probably a little bit more
pragmatic we're a little bit more
neutral we have our own models yes and
we think that they're going to be very
capable and but we also recognize that
different models will have different
sweet spots some are going to be really
good at managing data some are going to
be really good at translating languages
some are going to be really good at
reasoning and we expect that most
customers are going to want access to
not a single model that tries to do it
all but a range of models that are good
at different things and I think today
we're the only place where you can take
your own data and we have customers with
exabytes of data you'd be surprised how
many customers have exabytes of data on
AWS and they can take that data that
they've invested in and they can use it
with these models to create a net new
asset for their organization right that
is valuable and unique and private and
you can only do that on AWS so I have a
note here and it's just like man you
guys did Alexa first so why didn't you
end up leading this llm conversation I
mean the fact that like we thought we
were going to be talking to intelligent
assistants everywhere is kind of an
Amazon idea I mean it was an apple idea
but their execution was bad yours was
better you guys understood that we'd
want to be talking to computers and yet
you know we just mentioned that you have
your own llm your your own model that's
right there for people to access in
Bedrock trust me when I spoke when I
said I was going to be speaking with you
most of the general public was like
what's their strategy where's their
model you have one so just talk a little
bit about what happened there well I
think uh number one we're very happy
with Alexa Alexa is available to
billions of customers and requests
across hundreds of millions of endpoints
so listeners with one of those devices
in your home I apologize we apologize if
we're triggering uh the the woman or the
man if you've got it set that way so I
think we're incredibly incredibly proud
of the progress we've made with Alexa if
you look at the way that it has been
used in the real world the number of
endpoints that it's available in if
you'd have told me
six seven years ago that you could put
Alexa in a microwave right put Alexa in
a car
everywhere
it's a great idea
don't put your echo in the microwave no
do not do that just to be clear right
Alexa the service can live inside the
devices should stay outside okay good uh
so yeah I think yeah we're incredibly
proud of that I think our vision for
Alexa that we've always been striving
for has been to provide a per a truly
personal assistant yeah we talk about
this idea of like the inspiration of
coming from Star Trek and talking to the
computer and all those sorts of things
that's great but I think our actual
long-term vision is a really personal
assistant not a All-Seeing controlling
it's not that Alexa should have been
this it's that you had visibility into
what this could be yeah and didn't
release a Chachi PT first so was that an
oversight or well
only one person release Chad gbt first
everybody else didn't so I think that
that yeah I think we should take
inspiration from that it is a fantastic
probably one of the most remarkable
technology demonstrations that I've ever
seen I put in a demo
it's a great research to develop beyond
that
uh there are plugins inside chat GPT
right now that allow you to do some
people have access to plugins there's
code interpreter that does some of the
codings that your models are are working
out which is like speaking to data that
it's not trained on and saying bring me
some results well code interpreter
actually just executes code which is
generated by the model it doesn't have
access to net new information doesn't
have access to your own private
information and on that point nobody is
putting their private information into
chat GPT there are hundreds maybe
thousands of cios that are telling that
whole organization not to use chat GPT
because what's the concern there yeah
the concern is accidental data
exfiltration that you when you're using
chat GPT the the service that exists on
the web that we've all played with
whatever you type into that is being
used to train and improve the models
which makes sense as a research tool
that makes a ton of sense but if you're
a an organization and you start to want
to reason and understand or develop your
own IP that IP is not differentiated
against and it goes into the model and
it gets exfiltrated and we've seen
actual customers see their own IP come
back to them from the model and that is
terrifying to Enterprise customers where
the IP is the crown jewels and so
it is it's I don't mean it in a
diminutive way it is an excellent
technology demo it is an excellent
research tool from an exceptional
research company I'm not taking anything
away from what they've achieved and I
think they will continue to do wonderful
work however it is not the way that most
organizations in my opinion are going to
actually build and develop their
generative capabilities and that's your
bet is to help them build these models
and do it in a I imagine privacy is safe
we're not making the picture but I'm
just trying to figure like navigate this
conversation and figure out what you
guys are doing yeah and that's what it
is if a company comes to you and says we
want to incorporate llms in our business
you help them shape that and so they
don't end up dropping everything in chat
GPT and correct having that spit out to
their competitors correct and the the
models have a similar level of
capability so you're not really losing
anything there but what we provide is
security what we provide is privacy so
none of the information that is used
with our Bedrock service is used to
improve the underlying models in fact
none of that information even leaves
your network you can see exactly what
happens to that data and where it goes
and then when you want to improve those
models and customize them specialize
them for your own use cases internally
you don't want that specialization to be
available to your competitors you want
it to be private and secure and so we
make it really really easy to specialize
those models in a way which gives you
leverage against your own data and
allows you to do that in a way which is
completely private what's the business
opportunity there because let me let me
just tell you Amazon I'm sorry
Andreessen Horowitz
I just read they think that cloud on top
of AI the opportunity is 10 to 20
percent of this Genet AI spend
they said that sounds small what do you
think I think that uh two things if you
look at the broader opportunity I think
that uh this technology is as
transformational as the very earliest
internet and the web browsers that
allowed us to access it and it was that
early internet and those early web
browsers that gave inspiration and
motivation and growth to companies like
Amazon and Netflix and Airbnb and I
think that there is going to be a wave
of similarly Amazon size companies that
evolve out of the generative AI
opportunity generally and so I think
we're going to see multiple Amazon size
organizations develop and grow over the
next
who knows 20 years 10 years 10 weeks
it's anything seems possible and you're
content with 10 to 20 percent of that
next boom or you I don't know if I don't
know if that's true it seems very low to
me but I wouldn't be at all surprised if
just the uh AI part of our cloud
computing business was larger than the
rest of AWS combined in a couple years
okay
no consumer product coming out from you
guys
there's nothing to announce today that's
for sure big smile uh why not do it
uh do what a chat GPT style search I
mean we don't really operate in the in
the search space uh we don't have a a
deep investment in web search I mean you
have voice Computing it doesn't have to
be search
I'm just I think you can expect for sure
right uh to see new uh invention and
Innovation coming from Alexa and devices
and our retail stores and our ads
business you know there isn't a team
that I've spoken to at Amazon in the
past you know six to ten months that
isn't really focused on you know
understanding this technology and where
it can be applied to their own business
inside the company let me ask you this
Microsoft recently said that they have
11 000 customers who are using their
open AI
software building service
do you think how important is it for a
company to establish a lead in this
moment and
can you sit here and tell me what the
straight face that Amazon is ahead of
Microsoft
well um
number one it is incredibly early and we
are three steps into a marathon race and
I don't think anybody
without a smile on their face could call
a winner three steps into a marathon
Race but Amazon had this amazing moment
where it gotta now it wasn't everybody
going at the same time and you were very
early in AWS so you know this but AWS
kind of ran away with the cloud
computing field or cloud services field
until other companies started to figure
it out and by establishing itself so
early built that market dominance but I
I'm curious if you think we're going to
see something like that in this moment
or it just doesn't apply I think that
there's going to be multiple very
credible options for Builders and I am
strongly convicted that AWS uh if it is
not the leader after today's
announcements which I would say it is
you argue that it would be but even if
you take that as red I think it's going
to be hard to argue that by the end of
the year you know you won't see AWS
it will be hard to argue that AWS isn't
in the top one or two providers right
okay let's talk you might enjoy the
segment a little more let's talk a
little bit about I enjoyed that quite a
lot okay
well actually I'm going to go back to my
page of difficult questions
let's talk a little bit about building
right we have we're in front of a room
of developers at least I hope or other a
really very serious big technology fans
so thank you for showing they look like
Builders I mean that with the greatest
respect
yeah it's great so
talk practically about what people have
already done with with your technology
I mean you you had an announcement today
that that's kind of interesting about
agents right where we can build they can
build agents so I'd like to hear a
little bit more about like the Practical
level of maybe you can go step by step
of like what and briefly but like what
people would build with the AWS Services
sure uh I think the ones that I've seen
that are the most compelling
um uh number one just generative
responses so the sort of blog posts
advertising copy you know 3D meshes
those sorts of things where you're an
expert and you just want to you just
want a starting point uh you just want
instead of starting with an empty Word
document just give me a first pass and
let me iterate on it it's way easier a
huge Time Saver you do that all day long
very very popular the next area which is
less sexy but in my opinion maybe even
be a larger opportunity is using this
technology to improve search results
improve ranking relevance
personalization those sorts of use cases
where you don't even know that you're
working with a live language model it's
all in the background but they are
remarkable at boosting the uh the uh the
accuracy of those sorts of results then
you've got knowledge Discovery so that's
the sort of chat bot example and the one
that I'm most excited about is
collaborative problem solving so working
with this is a bit more science fiction
but I think we've materially Advance the
state of the art this morning with our
agents announcement where you are able
to as a as an individual or another
artificial intelligence system interact
with an artificially intelligence system
to solve complex problems that is a very
interesting area talk about what that
means well it means that imagine you've
got a uh any sort of business problem
that you can imagine
super simple I've got a thousand dollars
I want to turn it into two thousand
dollars how do I do that uh you may have
a set of artificial intelligent
capabilities that will help advise you
as to how to turn that one thousand
dollars into two thousand dollars and
you can interact with them one by one
and build the strategy yourself or you
can have them operate as a swarm of
Agents collaborating with themselves in
order to be able to build the best
possible strategy and for each like a
to-do list and for each item on the
to-do list they can recommend the
specific tasks that you need to go do in
order to be able to complete that so
similar like the baby agis that sort of
idea exactly yeah that auto auto GPT
approach of using large language models
to rationalize with other large language
models and where we see this doesn't
freak you out a bit I don't think it
freaks me out no I think we've seen
tremendous opportunity but here's why it
doesn't freak me out because it works
best and highly constrained domains
where you put so many constraints around
what it is you're trying to solve that
all of the agents none of them are
running a mock none of them are running
off and doing things you didn't tell
them to do you put the constraints on
and constraints are probably the single
largest force that we have to improve
the capabilities of these llms give a
practical example practical example
would be in my one thousand dollars to
two thousand dollars example you can
strain it to a set of markets you can
strain it to a set of stocks you
constrain it to a set of financial
products you can strain it to a set of
operations and buy and sell operations
that you would do in a particular given
of time and every layer of constraint
that you add reduces the chance that the
language model will just create a
spurious erroneous output but also just
keeps the whole thing grounded and keeps
the whole thing focused and when we've
looked at these approaches inside the
company you know they end up kind of
acting like humans like they argue and
sometimes they get stuck in a loop and
you need to go in there and intervene
and other times you need a tiebreaker
because there are there's two equally
good ideas and two equally good options
and you need some you need another agent
or person to come in and tie break and
so it's very interesting watching these
very tightly constrained
[Music]
ants run around trying to build things
on your behalf and you think this is
going to be useful
absolutely because it drives levels of
automation for solving complex problems
either completely automatically in cases
where you would want that or in tandem
with one or more people where you would
want that yeah let's talk a little bit
more you said there's some chat
applications you helped Bloomberg work
on Bloomberg GPT which is their chatbot
that queries financial data so talk a
little bit about that process like is
that the same are they coming in and
saying we're going to pick our model but
we're going to use your software to fill
in the blink yeah that's right so
Bloomberg has like a lot of customers
actually they have huge amounts of text
information and so they were able to
take all of that text data that natural
language data from market reports and
analyst reports and everything that
they've used everything they've
accumulated over however long Bloomberg
have been operating 50 years I don't
know and they're able to take all of
that and then load it into the cloud on
AWS and then use a machine learning
algorithm to build their own chatbot
which is Bloomberg GPT and they ran all
of that inside our cloud computing
infrastructure interesting so where does
Amazon get paid in that Loop we get paid
by providing compute capacity to
actually do the model training and for
providing the access to the large
amounts of storage that are needed to
store the data and then get that data
into the machine learning models and so
we provide that capability on a
pay-as-you-go basis as if it was a
utility and so you only pay for what you
use and we meter it by the second so for
one second you pay us a certain amount
yeah so this sounds pretty expensive to
me I'm curious I mean and it does seem
like it's the province of companies that
have more money I mean the fact that
we're talking about Bloomberg I mean is
sort of okay that's sort of indicative
of the companies with the resources to
build these models so tell us a little
bit about the cost factor and you know
who can actually afford the stuff yeah I
think training net new models is not
going to be very common it's uh it's
very complicated it is expensive to your
point you need a lot of compute capacity
a lot of data a lot of expertise some
folks that have differentiation in one
of those three things will want to
invest there I think it makes sense but
the vast majority will not want to
invest there now that said we want to
from the Amazon side make it as cheap
and easy as possible to train those
models in the first place and so we've
been investing in custom silicon in
order to be able to accelerate that
process with chips that are specifically
designed and built for
um large-scale machine learning training
and then once you've got the model you
want to be able to operate it in as low
cost as possible and whilst a lot of
focus is put on training if you think
about it you may train a model once a
month once a week let's say but you're
going to be running predictions and
inference and chatting with that model
hundreds thousands tens of thousands of
times a day and so if you're not careful
actually the vast majority of that cost
isn't in the training although it can be
expensive it is in the operationalizing
of the model for actually doing the chat
and that's why we have a second chip
which is specifically designed for
low-cost low latency inference important
friendship and I'm paying for the
compute on that what do you think the
cheapest is for someone who wants to
build their own part with AWS like
what's the entry level price uh if you
use an existing foundational model
I don't know 10 cents really
per just to build it 10 cents to build
it and then
the the cost of running it is priced per
token yeah I was speaking at Michael
schmulik from Bernstein he's a financial
analyst he follows Amazon closely and I
was like Mike what should I ask and he
said well look
this is going to cost a lot of money
Microsoft just said that they have
they're making something like a three
billion dollar infrastructure investment
in this is Amazon thinking about making
an investment anywhere in that range or
has it already
well I I don't think we're going to
release the size of the investment but
when you're thinking about the size and
scope of possible Investments they don't
get much larger than custom building and
fabricating chips yeah and so that that
is a huge investment that we've been
making at AWS for nearly a decade now uh
you know we're on our second generation
of our inference chips we're on our
first generation of our training chips
we're going to keep investing in those
and we're going to see I don't think
we're anywhere near the point of
diminishing returns in terms of the
capability and price performance
improvements that we can provide through
those chips and so uh I say I don't know
what they I don't know that the raw
number is actually all that interesting
what's more interesting is what's the
outcome and is that outcome truly
benefiting this broad democratization
that we're seeing okay so you've
mentioned chips we've talked about your
own model I want to take a break quickly
and then come back to talk about those
two things and I have another posted
that I'm holding with me and the the
headline is fun so I look forward to it
sounds great I'll be back right after
this
and we're back here with Matt Wood he's
a VP of product at AWS focused on AI
so you mentioned that you have your own
models your own llms and that's actually
something that's available if people
want to build
within uh Bedrock they can pick it's
Titan Titan or they can pick something
from open AI or lamma anthropic or AI 21
Labs we had a cohere this morning yep so
why build why build your own I mean it
seems so good up until the point where
you start building your own model and
now all of a sudden you're running into
the same problem that we talked about in
the first step that Microsoft has where
like if I'm building something you know
I I don't know if Amazon really is
neutral so talk a little bit about why
you built your own one institutional
looks at the same thing too he's like
you don't need an 81st model so why did
Amazon build it well on the 81st model I
think you do need an 81st model right
now like it would be completely
arbitrary to decide right now at this
point in time that we need to limit the
model or that we've got enough there is
so much opportunity it is so early I
think there's going to be no end of
invention in the in the foundational
models going forward so that said that's
why we took our approach of making all
of them available because who knows
which one is going to have a breakout
capability who knows which one is going
to be the best fit for a particular use
case our approach has been that we have
uh training we've trained a set of our
own foundational models we have a
language model and we have a vector
embedding model and each of those models
is actually a family of models so the
customers can choose the right model for
their use case not just for the
capability but also for the latency and
also for the price and so you may have a
use case for very very very for a very
simple small model
that you want to operate with very very
low latency and so that's an option that
you have with Titan you don't have that
option with some of these
extraordinarily large models
particularly that are hosted in who
knows where where the latency is just
it's just what you get but with Titan
you can choose the right trade-off for
capability and latency you can choose
the right trade-off for capability and
price and for each of those models you
can add your own data to the model to
improve it privately yeah now I'm going
to get to chips but it's always at this
point in the conversation like we're a
little bit more than halfway in or
there's always a thought that pops up in
my head which is we've been talking
about generative AI wanting to talk to
computers as a given as if we actually
want to interact with them in natural
language we want to have them as chat
spots we want to talk to them
but even the Alexa example shows that
like there was all this there originally
was this whole range of things we wanted
to do and then the test narrowed and
even I don't know what you're actually
let me ask you are you using llm like
chat GPT and and consumer uh Bots as
much now as you were at the beginning
I use we have an internal Tool uh that
we actually built for ourselves
primarily so that our engineers and our
own Builders could get familiarity and
practice with prompt engineering and so
uh it's super simple internal Tool uh
it's called the llm playground
and it is literally a playground you can
build little one-page mini apps where
you can provide a prompt and you can
chain that prompt into another prompt
and you can play with the parameters and
the different models and then you can
arrange the widgets on the screen to
build out little applications you can
build a chat application that way you
can provide it a URL and it will fetch
the website and then use that as the
part of the context for the prompt so
you can reason and ask questions about
the the website you're using this stuff
so so okay it's great that's good it's
the most fun I have all week honestly
really awesome yeah how many hours do
you spend doing it uh how many hours I
don't know I probably spent at least 30
minutes a day just exploring what's
capable and exploring what the team is
identifying as kind of emerging
capability okay so now that I've
effectively sabotaged my own question
for like the last minute and a half I'm
gonna ask it hit it um which is is this
something that people actually want to
do like do they want to talk to
computers I mean it sounds good it's
really freaking cool when you use it but
even now people are saying chat GPT is
getting Dumber largely people believe
that's because the novelty has worn off
so we talk about this generative AI
moment that you're saying it's going to
be bigger than the internet
um
what makes you so convinced that that
this time is for real what makes me so
convinced is that uh the level of uh
invention and efficiency and automation
that we've seen inside the company and
that our customers are experiencing uh
chat being just one modality so what
else do we have well we have the other
ones that I mentioned earlier like the
generative pieces the search pieces but
the collaborative problem solving pieces
the automation pieces completing complex
tasks that I think is where the majority
of the value is going to be and I think
that chat is a great user interface it's
a great way to explore uh some knowledge
and a domain it's great for new users
that are getting up to speed on a
particular product all of that so it's a
really useful use case but it is very
hard for me to imagine that we nailed
the use case first time right out of the
gate with chat I think that there is
going to be all manner of model
improvements which and supporting engine
improvements that allow us to deliver
customer experiences that we just
haven't even imagined yet and we are
doing that imagination and going through
that process inside the company now I
like a lot of our customers at AWS and
it is inspiring anything cool from
inside Amazon you can share I think the
uh some of the early stuff that we're
looking at we announced today around
generative bi uh so being business
intelligence thank you to be able to ask
and interact with your data just using
these
um uh using a chat interface one but
also to be able to create dashboards to
to be able to understand and find
insights number three and then number
four when you found those insights to be
able to quickly just summarize your
narrative immediately create like a
business report that includes all of the
summaries and all of the reports and
charts that you may need and then email
that around to your to your colleagues
like if you imagine the level what you
would have to have done before these
capabilities were available to be able
to people that you would have needed to
have teams of business analysts to
connect to the data they would have had
to spend time you know investigating
what to build and then building the
dashboard and setting it up and
then you have to train everybody in
order to be able to do the uh do the
work with the data and then you have to
find the insights which is very very
difficult it just shortens that whole
path to Discovery through Automation in
a way which is unprecedented so who's
learning from who here and please don't
say both is it AWS learning from the
rest of the workflow inside Amazon or
people inside Amazon learning from AWS
uh
uh I think it's honestly true that I
mean Amazon is a very big company right
uh I think we're taking inspiration and
we're kind of organizing ourselves
internally deliberately to take
inspiration where we find it and so
number one we're enabling all of our
Builders all of our software engineers
in which we have a pretty large number
to be able to experiment and try out
Live Language models through bedrock so
everybody has that capability and then
we're finding ways that they can show
their thinking and their invention to
each other with something as simple as a
demo day so internally we have multiple
different teams not a lot of them but
multiple different teams and they will
proactively reach out and we have a
schedule of people that are bringing
their demos and sometimes it's just
slideware sometimes it's an idea but
more often than not it is running
software that we can take a look at and
they share their thought process and
their implementation techniques and that
gets everybody else excited and then the
next iteration we're building on top of
that and round and round it goes and so
those kind of new idea nucleation points
across the company have proven to be
very inspiring to our developers and
number two enables us to share our
knowledge and our Discovery and our
thinking very very broadly and number
three honestly
prevented us from building the same
thing let's say we've got a thousand
development teams you start them all off
from the start line at the same time
come up with the same idea they're going
to come up with the same idea a lot in
the world so we've avoided that as well
let's talk briefly about chips uh it's
interesting because I think that there's
minimal awareness that Amazon has a
um its own llm in Titan and there's even
I think less awareness in the general
public I'm not talking about the folks
sitting here I'm sure they all know
about it but we hear so much about
Nvidia chips I I swear I feel like every
day I hear Nvidia like it's just a
marching you know chant in my head and
video chips and video chips and video
chips but you have your own chips uh how
are you making them and are they serving
the same purpose or something slightly
different
uh well we uh we acquired uh this is
offered chips for training AI that's
right that's right uh we acquired a chip
um design company called Annapurna uh
probably about 10 years ago now and
since then we've been on a path to build
uh arm processes for general purpose
Computing we have arm processors
specifically for high performance
Computing and to find very specific not
a large number but very specific use
cases that we could accelerate in
Silicon and the Machine learning work
use cases very quickly Rose to the top
and so we've been investing there in
terms of building out custom silicon
that you can deploy on AWS today for
building out your own large language
models and for running the inference
against them you design your own chips
as well that's right yeah and it's not
just arm that builds it right arm's just
a blueprint it's a starting point but
but with trainium and with in Friendship
those are totally custom designed who's
building it
um I think they're constructed in Asia
somewhere I don't exactly know where
Taiwan most likely yeah okay let's go to
the fun posted because I feel like
people are getting restless okay
uh no I'm nervous no it's all good I
think a fun posted for you is a nervous
inducing poster that is how it should be
oh ethics all right uh so you have your
own llm Titan yep what was it trained on
and how can I be sure as a writer that
it wasn't trained on my work
Titan was trained on publicly available
data and that's a very squishy phrase
oh it's very precise okay uh and data
which a proprietary data that we had uh
licensed specifically for the purpose of
training okay so can you say
definitively
that there's no chance that this model
was trained on like for instance
substack articles
uh if they're publicly available there's
a good chance that they were part of the
web crawl that we would have used uh if
they were part of or had been licensed
to a large proprietary set of natural
language we would have licensed that
with the right permissions to be able to
use them for training so my subset
stories are publicly available they're
available on the internet I guess I
didn't really opt in for them to be used
as part of training should I have the
ability to decide whether or not they're
going to be part of llm training or not
even though they are live on the web
it seems like I mean it's your content
you own it you can do as you please
um but I think it seems like a strange
use case to identify and single out
um who's to say what this data can be
used for when it's publicly available
you know you chose to make it publicly
available if you want to put some
permissions around it or you want to
take it private yeah that's totally up
to you you still own the data right but
it is public and as a result it can be
used for things that you may not have
thought through initially or that
weren't possible early on yeah and I'm
not sitting here and saying you know you
how dare you take it no I understand I
understand but it is it's a trade-off
it's something that people who are
producing content are having have to we
always thought it would just be Google
for instance that record all our stuff
but clearly it's going to be more and so
yeah I mean there are there are open
source open web crawls available
as open data for example you can just go
and look at what's in there it's
maintained and kept up to date it's
called the common crawl you can check
that out
Sergey Brin is back inside alphabet he's
called generative AI something like the
most exciting technology or moment of
his entire life
Jeff Bezos do you know what his feeling
is about this stuff
should you
I do I think he feels that it is I
wouldn't want to speak on his behalf of
course but you know I think he feels the
same like this is the single largest
transformational step in how we interact
with data and information and each other
you know since the the very earliest web
browsers and so I think um I probably
stole that from him at some point how do
you think you would feel of people
inside Amazon started using generative
AI for their six pages
uh that ship has sailed I can tell you
people are doing it for sure yeah what
of course okay but hold on it's a
starting point wait a second because the
whole point if I have it right from
Bezos is that
when you write something you have to
think it through super deeply and make
sure every idea connects one to one if
you turn that over to AI you're not
really going through the process well I
don't think that's true because what
you're getting back is just a first
draft and so that's actually a really
good way to explore your idea you can
get a gut check as to whether your idea
kind of tracks whether it's got legs and
you can start to poke and prod at the
idea and all of our ideas get better
through that poking and prodding and the
discussions that we have around their
ideas and so to be able to do more of
that early on you actually front load a
lot of the product development work and
you can do some of that with your team
you can do some of it on your own do
some of it you know with a with an llm I
think it makes perfect sense it's a huge
efficiency game can you when you read
one of these six pages written with an
LM can you tell that it's been involved
in the process I don't know if I've read
one that was completely written
autonomously with no edits even with a
little bit I think that I I think for
sure for sure that I have read
paragraphs maybe even pages that were
automatically generated with uh with
probably some pretty heavy editing that
I did not notice
that's good
it's very encouraging two more for you
okay
Amazon's culture the whole point I mean
I wrote a book the title is called
always do one so
the point of the book is that the
company operates as if it's a startup on
its first day and the culture has been
extremely intentionally built that way
by Bezos and there was a story recently
about how Amazon has more of a big
company feel
lately and you even have
um Adam from AWS I want to make sure I
get the language right saying uh you
know
we're going to be insurgents and you
only say this word we're going to be
insurgents when you feel like you need
that rally and cry
what's what's the story
uh I I see your theory it's an
interesting Theory
um I personally have not seen any well
number one I don't really know what big
company stuff I've only ever really
worked on Amazon and so I've been here
for for a long time I'm pretty well
entrenched in in the culture
um but I haven't seen many elements of
big companies slowness uh I haven't seen
many elements of big company politics I
haven't seen many elements of big
company uh in frugality or wastage and
so I think I'm sure there's many more
that you you could list off that would
be qualities of big companiness that
would be negative uh what I have seen is
like a continued focus on working
backwards from the customer and what I
have seen is like a continued focus on
scrappiness and a continued focus on
doing what we need to do in order to be
able to solve real problems on behalf of
customers I think you'll see that in our
approach to gen iterative AI you'll see
it in our approach to analytics and
satellites and all sorts of things yeah
because you I mean you really need that
sort of scrappiness if you're going to
be able to compete I mean I I
I feel stupid even saying this out loud
someone who's worked at Amazon for as
long as you have but this is going to be
a fight man like it's gonna be a very
very interesting time for sure and you
know it I would also say that the sort
of cultural norms that we'd have they
don't uh they don't exist and they're
not maintained without some energy and
without some effort right and uh anyone
that has had a um a a a conference call
with me from my office will have seen
the behind me on my office wall I fly a
pirate flag Parliament homage to Steve
Jobs you're a pirate okay the pirate
flag when I was writing the book I heard
about this pirate mentality yes and I
could never nail it down talk a little
bit about it yeah this is it's part
homage to Steve Jobs and the early Mac
team like very early day one drove tons
of transformation I'm a huge fan of
Apple huge fan of of the Mac I've used
Max all my life so part of it is an
homage to that but part of it is in
periods of discontinuous change change
you just can't operate like a big super
tanker you've got to operate like a
small merry band of pirates that are
just cruising and adventuring around
every Cove that you can think about and
just staying Scrappy and Nimble and so
for my part such as it is I fly the flag
in my office as a reminder to myself and
any of the teams that I'm working with
that this is a period of discontinuous
change and this is a time in which we
need to be Scrappy and resilient and
explorers and missionaries and that
people are listening
I so far so good people people seem to
like my flag all right uh this is not
the last one but I have to ask you about
this uh
the news is is gearing up for the FDC to
bring up a lawsuit to break up Amazon
obviously it hasn't happened yet it's
all speculation but it seems like it
will and it's going to be like
potentially the biggest government
action against a U.S company since
Microsoft maybe even my bell so do you
think about that at all is it even
something that you pay attention to that
one is above my pay grade okay
last question for you
you know you have a very interesting
position within Amazon because you're
like really working industry by industry
and helping them imagine how they're
going to transform with the latest
technology but we're in the middle of
this really unbelievable moment in
technology where we're starting to
really get a chance to imagine things we
couldn't before one example you guys
have released a medical note-taking
generative AI Healthcare application
Health scribe so I'm a son of a foot
doctor and my dad spent two big too big
of a chunk of his life writing notes
and just think about all the hours like
you could have had back uh I went to med
school before I joined that you know
yeah and it's just it's think about and
how much better care
you can provide to patients if you're
actually focused on that versus I agree
doing these things at generative AI
applications can do so why don't you
take us like on a top three interesting
things in different Industries
that you could imagine generative AI
having a real impact and I think that's
the medical one is interesting that's a
really good one what else where are some
other examples that we're just not
looking at yet
well number one I am sure that
everything I'm going to touch on that
there is a startup or even a large
organization out there already working
on it and they can they'll probably be
getting ready to ship as we speak
um there's just so much investment and
activity happening on this area
I think that a couple of spring to mind
uh the first one is cyber security there
seems like such an opportunity to employ
these language models in the
identification of the very subtle
signals that have become harder and
harder to identify which indicate some
sort of vulnerability or threat and so
being able to identify those threats
across multiple different sources with
better Precision is going to be better
for everybody I think that's one it's
not really an industry but I think it's
one that's going to be important it
still counts okay good I think another
one is just going to be developer
productivity like code generation we
didn't really talk about that yet such a
large accelerant I think there's going
to be elaborate talk about that with
with CEOs they're like yeah we're doing
it but they haven't really seen the
productivity increase now I know you
have seen it internally but we for sure
have seen it internally we've heard from
our customers yeah it's also early and
it wouldn't be at all surprises if
customers are still trialling it out and
getting a sense for it developers are
very uh very used to a particular
workflow and changes to that workflow
can can take time to get right and they
should be thoughtful about it but I
think that's another one another one's
really that's really going to drive
change and then just more generally I
think any industry
that has access to very very large
volumes of text is going to be the the
first places that we see this sort of
change and what's interesting about that
is there's some areas like healthcare
that are steeped in natural language but
they don't usually have like the best
reputation for being at the Vanguard for
technology adoption we're seeing so much
interest and so much excitement
healthcare lawyers legal Health Care
Life Sciences clinical trials drug
Discovery all these areas where
Financial Services Insurance like the
oldest stodgiest industries that you can
imagine they've got so much natural
language it's such a large opportunity I
think that's where we'll see the the
earliest returns potentially on
generative AI Matt can you believe this
audience I mean what a great I love
these guys listeners at home uh what
we're looking at is it says silent disco
type of conversation where everybody is
wearing headphones we're not even
using any Amplified sound at all and
just been looking at this crowd as we've
gone and they've sat here and hung on
every word so yeah thanks to you guys
thank you so much thank you for being
here
and uh I'm gonna thank Matt in a second
but I'd be remiss if I didn't mention
that the big technology podcast airs
every Wednesday and Friday Wednesday a
flagship interview like this Friday we
cover the news all right everybody thank
you so much thank you to you thank you
to Matt thank you I appreciate you enjoy
the rest of your day thanks
it was awesome
[Music]
[Music]