A Piece of Pi: Embedding The OpenClaw Coding Agent In Your Product — Matthias Luebken, Tavon

Channel: aiDotEngineer

Published at: 2026-05-11

YouTube video id: vAIDdLKB6-w

Source: https://www.youtube.com/watch?v=vAIDdLKB6-w

All right. I was introduced to Pi by uh
um looking into openclaw. there was a
conference uh a meetup and said like
okay we're doing open claw and I wasn't
so much interested into like all the
craziness things that people are doing
but I was more interested in
understanding
uh of how these things work so I was
looking into pi and you know uh
understand the the whole world of what
pi is able to do um this is the one
picture you need to take please feel
free to take more pictures uh but all
the slides and the examples are there.
Uh so that's the one slide. All right.
Very quick uh about myself. Uh we're
creating a small company uh TAI. We're
building agents for uh organizations
small out of Europe uh but getting
started. And uh what I really like um
about
sorry
uh what I really like about um uh
Mario's talk is this this quote. Uh you
probably have seen uh this this morning.
We are on the uh we are in the
around and find our own phase for coding
agents. Right? So everything that I'm
going to show you is what I know today.
Right? And u I'm going to do the talk
again in a couple of weeks and it's
going to be most likely be different. uh
but um as as Mario was showing this
morning um he has created this minimal
set right this this coding agent that is
available um uh for for you for you guys
to to fool around with and that's what
I'd like to encourage you
so coding agents and why is it so
exciting for us to build more products
this is Ken Thompson um inventor of uh
Unix and this is the famous quote by him
one of the quotes, write programs that
do one thing and uh one thing well and
um I really like that because that's
that kind of like works uh to our
advantage with agents and um the best
part where I show this is with co-work
so this is co-work uh cla's desktop um
and they're basically are bundling their
coding agent into something where they
feel is more applicable um and to be
honest I've seen very good receptions
around this and when you use it uh with
financing tools with their finance
tools. You always need to work with
Excel, right? So they have this Excel
skill down there um and it talks to
Excel, right? Well, it doesn't. Uh
instead it uses a a set of small tools,
small CLIs, um uh pandas, open pixl
stuff from Libra Office and package this
into their own skill uh to make it uh up
and running. And I think this is a great
example to kind of get your going, get
your thoughts going of what what is
doable.
Um, I haven't written a book and nobody
can write a book about this, right?
Because there are no patterns, right? We
need to figure this out. We're seeing
some emerging patterns in the coding
space, right? There's obviously tons of
different coding agents and we're seeing
this, but there's no authorative
resource around this, right? So get
going. One thing uh when I was talking
to Ivan yesterday uh we realized is like
one architectural pattern that we're
seeing is that make it easy for coding
agents right now that is very broad but
think about it right like like make not
don't try to be you know very um complex
and things but think about the the
coding agent uh what is it good at and
how do I build my system so that the um
agent is easy make it accessible
and I I have some examples. All right,
this is the rough agenda uh for the next
10 minutes or so. Um I'm not going to
talk too much about pi in openclaw. Uh I
have a two slides, slides are online, so
we'll take it from there. So again, very
brief uh introduction of pi. Um Mario uh
great work. Something he didn't mention
is that he's joining Arendelle uh which
I think is awesome. uh it seems like uh
you know great great folks working
together and uh yeah it's open source
it's minimal so it's it's just perfect
to get started and the other part that I
do want to reemphi emphasize on is is
give it a try right we're going to talk
about a little bit different but open up
pi and ask it to build what you want
right it's amazing of what it what it
actually is able to do by the system
prompt uh that Mario has shown.
All right, these are the extensions. Um,
so again, all the extensions you can
download, uh, build yourself or download
and yeah, ton tons to explore. All
right, so let's going this talk is not
about the coding agent itself. So using
it for your daily dev works, but what
can we potentially do with this? And the
starting point are actually not coding
agents, right? The starting point is um
and I encourage you to do the the same
is looking at the core agent itself and
there's other SDKs but you know we're
talking about pi so let's let's let's
use pi and what is an agent an agent is
actually just an LM agent that runs
tools in a loop right so you have some
goals you have some context information
agents MD uh in many cases and then you
do do coke tool calls right and you get
some results and you know you basically
do do it in a loop, right? That's it,
right? There's not not much more. The
rest is magic trying to put it in your
use case a little bit more in the other
use case a little bit in that direction.
So, that's really it, right? So, pretty
please uh uh don't like open the curtain
uh play around with it. Now, with agent
um uh uh agent core, this looks a little
bit something like this. You have an
agent class. This is all Typescript. Um
you can uh you know address all all
sorts of information information you can
prompt it um uh with different
information uh and um also you you have
an event system so you know a lot of
things that that that are going on. So
um small example uh this is a CRM lead
qualifier. I don't know I've started the
CRM use case for my personally and it it
just sticks around. So um terminal
interface obviously uh small uh
TypeScript application three uh uh three
files really easy and you can see this
right you have a couple of commands that
you can execute and you know show me all
leads and score them right so that's
what we do uh show all leads and score
them and here you see all these you know
things that are going on under the hood
right you see that that the assistant is
calling uh tools that you get some
results and eventually you know, you get
some input. Now, obviously, there's tons
of things to do, but you know, I've just
vibe coded this away uh uh uh and it's a
good again good uh learning exercise.
The system prompt um uh you know um as
you could imagine, right? You know,
calling out the different tools that
what you do, right? So, all pretty
straightforward if you are building an
agent. This is an example of how you
inject here, right? So um we said we
want we we do call tool calling right we
reach out to this uh and call a specific
tool but for the agent for for steering
it more right you know a typical hook
would be before the tool call do
something right and in this case we
don't want to update a contact uh
without you know checking something or I
don't know you can imagine any types of
authorative uh role based access
whatever enterprise feature in here but
basically you know uh just before the
tool call. There's another one events.
So, we've seen these, you know, uh uh
the stream and you might have seen a
little check mark there. Okay, the tool
call was was fine and returned some
result. So, again, we're subscribing to
events. All pretty straightforward and
again, please give it a try. All right,
so this is simple agents others agents
SDK uh are are available. Um and now
we're moving through the coding agent.
Now what's what's a coding agent? At the
end of the day, it's really the same
thing as we've seen before. It's a you
know normal agent, right? It runs tools
in the loop. But now we have a runtime
and some type of shell, right? Bash is
seems to be the uh the shell that that
everyone is using. But we have a shell
and a runtime to to start executing.
And now things are getting interesting.
And now the the magic of of what you've
seen with OpenClaw uh suddenly shines.
Uh um Peter uh shared this this example
uh on some presentation where uh he sent
a message to his open claw and sent a
voice message. Now at that time openclaw
um and I still don't know if there's any
like special plugin but at that time
openclaw didn't know anything about
voice about voice messages. So what what
it did is it it uh created and used
different tools
uh in the end one of the tools was uh
ffmpeg right on the local local machine
and it started this and this was one of
the tools right so from the outside it
it looks like learning
but in the inside it's actually just
another tool call that is available to
the agent and that's why these things
make it so interesting. So um again uh
the example here um now this is a little
bit more sophisticated
but the uh important part and and this
is the extension API and you know please
look it up online. We we're going to do
two things or the the things that I'm
most mostly interested is in in session
events and UI interaction and yeah uh uh
look it up online. But here's here's the
the actual extension. Now again this is
what you would in a coding agent you
probably just generate by asking it but
here if if we have a look um this is a
CRM uh TypeScript a small snippet of it
and basically what we're now doing is
we're doing the same example as before
right and we have a new command called
pipeline right so if you have the slash
commands and you have a new command
called pipeline and now we are able to
we're loading all the context um and uh
you see this little in um u don't have
the lines just below step one uh you can
see a context UI select right so all of
the sudden we're not only interacting
with the backend systems and and
sessions and and of those sorts but
we're also interacting with the UI and
we're able to select right and that's
that's got got me thinking um so right
so you have this this command and again
this is now just the coding agent Right?
We're not talking about the core agent
class, but but this is how you would
load up Pi if you just don't download
the the coding agent. And now with this
new extension, we have Pi, right? And we
can start selecting things, right? So
this is a simple simple select here. Um
and you know, you you even have
dropdowns. Now the important part here
is these are extensions and the
framework uh that currently pi um has
included is catered towards the use
cases of a coding agent right so we you
know there's lots of work and other
things to do to make this ready for
others for other types of applications
but I hope you can see and understand
the vision where where this is heading
and um yeah you know this is all
terminal Right? So you wonder how would
this look like in the web? Um it
currently is not possible if you ask Pi
to build something. So I ask Pi to build
something. Right? And this is the web
UI. It would be a web UI. Same command,
same selection, all based on the same
extension mechanism. Now um there's a
refactoring going on to make this better
accessible and make it more clean. But I
hope again it shows you a little bit of
of where the where the things are going.
All right. Now, um PI and OpenClaw um is
um is a special special setup, right?
So, PI and Open Claw what we have there
um is that that now we're not only
talking about like like um a single
agent in a single session in a coding
environment. Uh but now we have a
multi-channel uh environment where uh we
have um you know multiple threads going
on, multiple agents going on. So there's
a little bit more to it. Um this is um
and and the interesting part right
that's that's where where I got started
is is like if you look into um you know
the the packages um uh the core packages
of of of pi all of them are used in
openclaw right so open claw has this uh
uh this function run embed uh uh pi
agent and it creates a session right so
sessions um uh pi itself has a great
session support and It creates a session
agent and streams all the information
back. We have um the coding agent which
we just talked about. We have agent core
as um uh the other part that we talked
about. And there's two other uh minor u
or major packages pi for the unified lm
abstraction and a terminal ui interface.
Um there's um open claw has built its
own plug-in mechanism and that's because
um uh you know it's a different use case
right and has different requirements. So
you have plug-in support for multi-
channelannel routing, different or uh u
provider orchestration, sub agents,
gateway support, yada yada yada all the
things that you know by openclaw but
it's based around the core mechanics of
of pi and and and leverages it. Cool.
But uh one thing and that's that's
that's the like the the major gist I
would like to bring across is like okay
what do we do now with this? what are
other options for us to do? And this is
one of the applications we've been
building um for um for a client. Um and
basically um uh the the the use case is
a sales process um they get um uh
requests for proposals um of of an
ordering another um another system,
right? Um parts parts being sold by that
company. And we're taking all this
coding agent all all of that we're
taking away right where we we're new
fresh new thinking right and look at the
process from the get-go. So um an email
comes in right we we we monitor
basically that inbox then we have some
gateway because what we want to do is we
want to forward this to different agents
right so here I have um multiple agents
right uh the way it's structured is we
have one agent per customer and that
agent has a general harness right agent
MD uh um agents MDE as an example but
you can obviously also use different
ones and that helps um understanding the
role of that agent. In the specific case
it's it tells of how to use the system
and how to react to certain you know
inputs outputs etc.
Now um the other one is customer MD
where we basically explain the agent
like you know the specific customer
might have you know specific twerks
right specific um uh access specific um
um discounts and all of that sort and
then right and that's what I said like
earlier I I like using sessions then for
each case right we're we're creating and
reusing existing sessions so we can back
and forth um um know what what was
previously talked about. All right, so
email comes in, we're looking at the
gate um inbox and we route this to the
different agents and now we have tools,
right? So we have these different tools
uh to talk to the CRM to talk to the ERP
um and get the right information out of
the system for this agent to look on
like like behave like you know maybe it
has you know new contact information or
of that sorts and again we make this
available we make it easy for the agent
to access right and our way currently is
doing this with CLIs right so CLIs our
agents are really good at using CLI so
we make it available as a CLI I we put
we make sure that the data is secure. uh
we have our own sandbox and then we're
creating the drafts again right so
that's the system and I hope by this
point you basically understand like
logically where these things uh uh fit
together but how would this look like um
oh one uh a final thing right there's
always the question around okay
sandboxing etc and and to be honest
we're on the uh just on the on the steps
of of getting there but if you've seen
um Nvidia's announcement uh around um
open claw, their policy, their open
shell is really really interesting and
um um it's it's it's a way of it's one
ways of securing an an agent. We're
looking into this. Please do as well.
All right. So, how does this look like
um to to to kind of like get you an
understanding of of how these things,
right? So, here's the dashboard. Um
rather uh boring, but here's the in uh
the email the inbox, right? So again we
see the the email coming in and yeah we
um it's one of one of many emails most
of them ignored but this one is like the
the DLM call said okay I'm I'm
interested in this and it is associated
to a case right we see the case up there
now this case is again is an agent
session right uh so we find the session
and associated to it
um we then create a draft uh so there's
tons of calls which I'm going to show
you in a
But basically the output of all that is
a draft email that the user will be able
to use. Right? So our thinking is uh let
them user stay in in email, let them
stay in the the inbox and drafts and
they don't even you know need to do a
lot. So this is more like an admin
interface. They can stay in email but
basically the output would be a draft
generated. And how does that look uh
behind right? We we had the the
different sessions before uh the threads
and this is the same thing right the
assistant says uh well apologies to
German but uh now I'm looking at the
articles it does different tool calls
right it gets gets results and does this
in a loop to result right the end effect
for for the user is I'm looking at my
inbox there's a new email it's
associated to a case and I get a new
draft which they can freely edit but um
under under the hood we have all these
um uh agents working. All right, that's
that it is for me. Um again um here here
you find the slides. Um key takeaways
please. Coding agents are and will be a
core building block uh for your software
systems. I'm I'm betting on it. A lot of
people are betting on it. So please give
it a try. Pi is perfect for tinkering
whether you like it or not. It's
minimal. You can rip things apart and
put things together. It's perfect. So,
please go tinker. All right. Thank you.