Gateways are All You Need — Karan Sampath, Anthropic

Channel: aiDotEngineer

Published at: 2026-04-27

YouTube video id: CD6R4Wf3jnY

Source: https://www.youtube.com/watch?v=CD6R4Wf3jnY

[music]
>> All right. Um so everyone, I'm uh Karan
Sampath. I'll be talking to you about
how we Anthropic think about MCPs in the
enterprise. I've alternatively titled
it, I think more catchily, is why we
think gateways are all you need. Um so
before I go into the talk, uh I'm going
to quickly tell you a bit about me. I
was uh I'm a forward deployed engineer
at Anthropic, um
first one outside uh the US. Uh a lot of
my work includes working with
enterprises on things like MCPs, and I
also work on our internal use cases. In
this talk, I'm going to be
positing to you what we think the
problems with enterprises, what
enterprises face with MCPs today, why we
think gateways and the the necessary
implications that come out of it are the
best way to fix a lot of these problems,
and what is that how does that align
with our future vision for agent tech
deployments. Um so before we go on, I
know a lot of you already know this. I'm
going to quickly run through this, but
just a very quick overview of MCPs as it
is relevant today. The first is of
course we all know it's an open standard
that was created. Uh most of us know
that at Anthropic. Um there is an
official registry today which contains
over thousands of servers, and this is
growing rapidly over the last year. Um
and we see this happening that
individual companies all the time are
building new servers uh very quickly,
and are trying to ensure that they stay
ahead and trying to adhere to this
protocol nowadays. But still, even if
this happens, we see a major problem
when enterprises try to use it.
Enterprises struggle to deal with what
they believe to be table stakes. Things
like observability. When they want to
know who's using my MCP, who's using
these tools, how do I ensure that the
correct people are using it, um and how
do I know how to develop on certain
parts of the MCP protocol, which parts
of my tools aren't working properly.
These are something that something
that's simply completely opaque to
enterprises today.
Second is access control, something I
just touched upon already, right? How do
I ensure the correct correct users have
access to these servers? Things like
ensuring that certain servers are scoped
correctly. Tools are only made are only
allowed for certain groups of servers.
If you're working in observability MCP,
you might want the entire company to
have
view into like why which things are
failing, but only certain people to have
the ability to actually change things
and update new dashboards, right? These
are the kind of things that currently
are quite hard to do with MCP servers.
And honestly, not something that we as a
community have worked on enough. And
finally, security. And I like to think
of the three of these is almost like a
three-headed hydra in some ways, right?
Um security is a wide range of things. I
like to think not only in terms of the
MCP server, where you know you
enterprises want to know how do you
verify whether a server is safe, it has
uh the correct protocols and the correct
ways of ensuring data exfiltration
doesn't occur, things like uh you know
the tools on on can't be used in a
harmful way both for infrastructure
externally, and but your internal
infrastructure as well. But secondly,
how do you ensure remote clients that
are perhaps untrusted in nature can
access your private data as an
enterprise, right? These are all really
hard problems that you've kind of solved
in previous paradigms with APIs, but we
really don't have a good way of solving
it uh at the moment.
So just going back to the registry
point, it's really useful to think of
where we are and where we want to go,
right? Registries are really useful, and
I don't think there's any discussion
that here where we are having where we
think they're not useful. We're really
proud of it and we're really happy we we
helped develop this. But it's really
important to realize that registry
registry simply aren't complete for an
enterprise. And what's like funny about
this is is that MCPs were specifically
designed um if you attended David
David's talk earlier today, MCPs are
specifically designed because they allow
themselves to be so much more useful for
enterprises. And there's this kind of
gap which the protocol allows for that
we have yet to build into well, and
these include things like
authentication, but also access control,
observability, and credential
management. These are all things that
enterprises need and are critical part,
um but are simply not working. So now
that we have this happening, what is
what do enterprises do right nowadays?
What they do is something like this,
where every single team now with cloud
core, you know, and explosion of coding
across across various surfaces, can now
start developing MCPs. But suddenly find
out that they can't actually get them
deployed. Or even if they want to get
them deployed, the MCPs often can't use
the tools they want to, those tools
can't give them the correct access, and
security teams on the other end are also
pretty justified how they're going about
it, because they're often overloaded,
and they're often unable to see which
MCPs they want to go through, and which
they want to which they want to allow.
And finally, at the company level, CEOs
and C-suites are like, you know, why are
my MCPs not working correctly? Why are
my agents ineffective? Why can't um why
can't your agents actually, you know,
being be the thing that we all thought
it was going to be? And so this
bottleneck, which we're seeing right
here, is something we need to solve. We
need to ensure that security teams
aren't overloaded, that users are given
the freedom to actually develop their
own MCPs, and organizations have
visibility into all of them. And so what
I'm going to tell you is that this
problem where enterprises stay with a
handful of MCP tools is going to
fundamentally restrict the protocol and
hurt agents until we really go and solve
this.
And I think it's worth zooming out at
that moment, right? It's like I think
it's really valuable to think how
important these paper cuts are, and how
valuable it is to try and invest the
time to try and solve it um
super super well.
On to why we think the gateways are a
better solution. I think this is a very
way to good way to build intuition.
The core intuition here is at a point in
which almost all of your teams can build
MCP servers really well, or have the
ability to theoretically be able to do
so because they're simply using coding
agents that can understand the structure
of the servers very well, that are able
to understand what the tool definitions
look like, what you should want, uh what
access controls look like. The really
important thing for security teams and
enterprises that want to allow this to
be decentralized is they need to
establish a root of trust. And so we
think that the goal for a secure this
for any security team is to is to bless
one platform. And this is if I take one
thing away from this talk, I would
really suggest it be this slide, because
it's irrelevant like you can you might
want to use a gateway and I'll mostly be
talking to you through this, but I think
this intuition is really the intuition I
would really want to stress. Because
enterprises that are able to do this, in
our experience, we've done this
internally and done this externally,
too, are able to explore the usage of
MCPs, and thereby explore the usage of
how powerful the agents are. And it's
worth obviously coming back here and
saying like, you know, MCPs if the usage
is exponential, every good MCP you have
helps all of the agents in your company.
And so doing something like this has
knock-on effects beyond just that one
MCP. Um so now now that we think that,
you know, our platform is really useful,
let me tell you why I think gateways and
I'll define it for you are a good way of
doing this. So a gateway, as a black box
definition of it, is simply a middleman
or
sort of middle layer in between your MCP
servers, and they can be numerous into
the hundreds, and any MCP client. Notice
the diagram here is notice is extremely
simple, and it leaves a lot to be filled
in. But what I really want to talk about
is what we want to get out of the
gateway. Things like authorization,
authentication,
observability, ensuring that you have
correct connectivity between
clear secure connections between your
MCP client, which might be untrusted,
and internally. And also finally, you're
able a easy way to host and deploy any
new MCP server. What this allows for is
that any new MCP server now does not
have to deal with any of these five
things. And a team that wants to wants
to add a new server only needs to care
about what is the business logic. So
your legal team which wants to review
contracts only needs to care about,
okay, if a contract comes in, this is
what I want to be seen, this is how a
redline should happen, this is what this
is how you should escalate your
different people. They don't need to
ensure, you know, who who accesses this,
how often can it be accessed, how do I
know it's being accessed correctly, how
do I know it's like scalable, and if I
want to have new agents come in, how
These are all things you need to worry
about. And that's a really really useful
middle layer to have, because actually,
in the world we live in, your legal team
can build the MCP server on their own.
They aren't forced to go back to a new
technical team and build it, right? And
so if you really do want to live in that
world, a gateway is a fundamental piece
of that infrastructure.
What does a gateway contain? Now there's
like multiple definitions of gateways,
we're going to give you examples, but in
my mind, uh there is there are normally
these components which usually exist
within it. You know, I'm not going to be
like this is hard and fast, this is
neither exhaustive nor nor the only nor
like the without these like there's not
all required. But what you would really
like to do to achieve the goals I just
achieved I just talked about, is you
kind of want a way to do auth, you want
a way to do access control using roles,
you want a way to route using a proxy
such that any MCP client can only see
your gateway, and and the gateway then
can route to the individual MCP servers
which treat the gateway as the only
trusted endpoint. You want a way to
ensure you have a tunnel which is a
secured connection. You want to have a
sub registry, which is your MCP servers
internally. And finally, you would want
to have any additional tooling. Tooling
like a CLI for your gateway, such that
anyone who wants to create a new MCP
server in your in your company can
easily create one because the gateway
has a quick and easy CLI that not you,
not that team, but but the agent that
team is using, that cloud core or
something else, can easily understand.
Um these are all parts that when put
together become really powerful, because
someone who is just wants to create a
new MCP server can use the gateway CLI
and just easily integrate into these
five components, and then completely
focus on their MCP server, right? And
that's why we really think that in
making that kind of one-time investment,
which doesn't hopefully doesn't require
a lot of maintenance, and something you
can easily do with agents would lead to
several knock-on benefits.
Um just like and just like a
higher-level view of what this gives
you, right? So, we already talked about
this in terms of a listing of what we
think is required, but just looking at
once we have this, you have a vision of
a gateway which can give you access
authentication very easily. You might
have your own IDP which you can plug in.
You can have delegated identity in terms
of users and agents. This is something
we think is going to be really important
into the coming year, where we think
agents
are going to require newer and novel
definitions of identity that, you know,
you can uniquely define and think about
and scope for your enterprise using a
gateway. We'll ensure that you can have
one access control panel for all of your
for all of your agents and and MCPs, and
your access control can be scoped
depending on like whether a team is
accessing it, whether a user is
accessing it, whether uh whether another
employee is accessing it. And finally,
observability. And I think observability
here is nuanced in the sense that you
not only want usage metrics to reflect
what MCPs are
being used, but you also kind of want to
know how how are your tools being
defined. In a world where the MCP
protocol is itself being developed so
rapidly, you kind of want to see how to
how what are your load-bearing tools and
how do you better adapt them to meet the
needs of your various agents. So, I
think this is really really cool and
really useful. It kind of, you know,
sometimes some people often say that it
sounds too good to be true, but I really
encourage you to try and take a look at
this
and try and and try and see where you
can go from there. Um but let's say you
have a gateway now, right? We've already
talked about like the minimum
requirements. I think there's a lot of
exciting worlds that you get on very
easy follow-ons or almost free lunches
you kind of get from there. The first is
it's very easy for you to add any new
surface. So, you can easily have your
MCP servers now plug into cloud.ai, they
can easily plug into cloud core, they
can easily plug into cloud core work
because why? Because all of them are
kind of listening to the same gateway
and you kind of you only need to do it
one time. Compare that to the world
where you have 40 different MCP servers
and some MCP servers are better
configured for only one of these
services in one of these clients. This
is really important because this kind of
ensures that you can be kind of
invariant to any new surface that comes
up. And I know like this is something
that sounds interesting coming from me,
but like it's really really useful for
any new enterprise. It's really really
useful for any enterprise to do this.
The second is you have far more secured
connections that can be built in.
Currently, you know, your MCP servers
often have access to sensitive data
internally and you any enterprises are
worried about, you know, "If I send this
out to another client, is that data
going to be exfiltrated? What happens to
my data? How do I ensure that, you know,
my like literally years and years of
work doesn't go away?" You can now
invest in much more secured connections
between your MCP servers that are
between your MCP server and your client
that is completely encrypted
and can ensure that you have root trust
in how in what kind of data be sent. And
this is really important. I think like,
you know, we've often done you've often
made MCP servers with kind of play data
and things that, you know, enterprises
can't really use, but for enterprises to
actually derive the exponential value
they can get from this, this is
something we need.
The third is faster iteration. The real
value in like ensuring your teams can be
decentralized completely is going to be
that they can iterate and develop their
own workflows much faster. If the legal
team is able to very quickly just change
their legal MCP rapidly and iterate on
it without repeated security reviews,
that has a massive effect which can just
which just builds upon itself. So, this
is something that shouldn't be
underrated. The fourth is that you get
very much more standard primitives. So,
any new MCP server that you want to
build kind of has to adhere to
primitives that are that are within your
enterprise and how your enterprise wants
to go about it. A common request, you
know, a common problem enterprises have
is like, "How do I ensure said AI agent
meets my standards and operating
procedures?" A gateway is a way of
encoding that. A gateway is a way of you
being like, "These are my standard
procedures, these are the primitives I
want to see, these are the tools I
expect, and these are the things I don't
want to see." And that's a very quick
way of encoding how you as an enterprise
want to behave. Uh the fifth is
pluggable credentials, right? What we
currently have is certain MCP servers
that only accept one type of user
authentication, right? And it's very
hard to kind of make sure it it it
becomes
like, you know, you can have a
company-wide one or a team-wide one or
something with service accounts. These
are all things which are often useful in
different cases, and that becomes easier
with a gateway because a gateway
allows any new MCP server to easily
support new credentials and swap it in
and out in an intelligent manner. And
then finally, it's scalable, right? Like
if you have 40 MCP servers that need to
scale to from tens to hundreds to
thousands to hundreds of thousands of
agents, that's a really hard surface to
maintain. A gateway which is able to
take all these requests in and farm it
out in an intelligent manner is a far
better place for your teams to focus on.
Finally,
you know, this is not something that's
we think is going to be like we're we're
telling you out of the blocks. We have
examples of this both in the open
source, you have provided something
in-house, but this is also something we
can help with. This is something I do in
my daily basis. If you're interested,
I'd love to talk to you about this after
this.
Um but just finally zooming out, right?
I want I want to take these last few
minutes to think about where we're going
with this. I like pitched a version of
the MCP world to you that, you know,
sounds cool, probably works, but does
this really work in the hundreds of
things you've been hearing over this
conference? I think what we want to
ensure is the larger picture where
things are going is we want to separate
the agent harness from where your data
lives. We have many more surfaces coming
up. We see an explosion of agents, but
those agents shouldn't be tightly
coupled to where how your data
structured and how your MCPs are
structured. An example of this is just
the pure definition of an agent should
like when you have tens of thousands of
the orange box on the left, you don't
need to keep you don't want to ensure
that it has to be opinionated. You don't
want you don't want the things on the
right to be opinionated on how this
works. We have this working we have an
example of this which was released
recently, right? Just as an example of
this right now, if you have an MCP
gateway, you can use you can easily
connect it to cloud managed agents which
was released recently, but you can also
build it into use it internally
with your own cloud agent SDK as well,
right? And this is just an example from
the cloud.x ecosystem. But think about
the value of this. As an enterprise, you
now have the ability to quickly decide
which agents you want to keep in-house,
which agents you want to have outside,
but that becomes an invariant decision.
The gateway remains regardless. What I'm
trying to tell you is that the gateway
is an investment which will allow you
give you the flexibility to try to try
and meet the wide-ranging agent needs of
the future, and that's going to be
really really exciting.
And that allows you to not only
not have to think that much about agent
design at the moment, but it also allows
you to really invest very strongly in
opinionated MCP gateway primitives.
So, just in summary,
I would say three main things here,
right? The first and the most important
takeaway is to invest in common infra,
to not try and roll your own MCPs, and
to ensure your teams can build their own
MCPs. Second, we really think gateways
for secured connections allow you to
build that root of trust. And third,
that moves towards a world where we
think the agent harness is better be
able to separated from your data layer.
So, yeah, thank you so much. If you have
if you're interested in any of this, I'd
love to talk
and that was the talk. Thank you so
much.
>> [applause]
[music]