Knowledge Graphs in Litigation Agents — Tom Smoker, WhyHow

Channel: aiDotEngineer

Published at: 2025-07-22

YouTube video id: yYxr6LdXNWM

Source: https://www.youtube.com/watch?v=yYxr6LdXNWM

[Music]
Hello everyone. I am here to talk about
graph rag as we're here for the track.
But I'm talking about what to do in the
legal industry and what we do in the
legal industry and what does it look
like to turn documents into graphs and
use those graphs in the age of AI.
I tend to have to qualify why I'm at
places. Uh there's various reasons why I
could be talking today. Uh you choose
the one that you want to, but generally
I've been working in graphs for about a
decade. I have a good relationship with
the NeAJ team. Uh and I've been doing
graphs for a long time, but primarily I
am the technical founder of a company
called Yhar.ai and we find cases first
uh before lawyers do and then give them
to lawyers. Now, how we find these cases
is a process that I'll go through, but
we use a variation of graphs, multi-
aent systems, signals, etc. And I'll
detail through today how we do that at a
high level and a low level. And I'm
happy to answer questions at any point.
This is broadly what we do. We work in
law. This is an example. We find class
action mass cases before other people
do. Um, we have agents, uh, we have
graphs. We store that information. We
scrape the web. We qualify that with a
proprietary process. And uh we deal with
lawyers every day and understand exactly
how they think and build these cases.
And the cases I'm referring to would be
like many people used a pharmaceutical
product. That product has caused them
harm. Science has proved that harm and
we can collect those people and
collectively sue the pharmaceutical
company. So we support the law firms
that do that. And as I'm talking and
everyone here for a graph rag track can
start to imagine that I'm starting to
develop a bit of a schema there. I'm
describing individuals. I'm describing
products. Those products have
ingredients. Those ingredients have
concentrations. Those concentrations may
have an ID number and all of a sudden
you can start to imagine there is this
largeworked schematized bit of data that
has particular points in it that are
very valuable and very visual and very
useful to domain experts.
So I'm going to start to use some
definitions because there is knowledge
graphs have been around for a long time
and ABK would know that more than I
would. But I started my PhD and well my
masters in graphs in 2016 and it was not
nearly as popular as it is now and it's
fascinating to see how far it's come.
But I do think it's important for me to
define how we use them and how we think
about them.
Broadly to me, graphs are relations.
That's part of the visual element.
There's a backend element as well. But
it's the benefit of using graphs is that
I can see what is connected to something
else. I can be explicit about what is
connected to something else. And I can
do mass analytics on what is connected
to something else. All the way from I
can see it down to I can do large scale
analytics on it. Is the value of the
relations. And when I use relations,
it's not necessarily node to node. It
can be node to node to node. It can be
multihop. It can be as varied and as
forked and as distributed as you want.
This is why we use graphs in our
process.
Broadly throughout the process of
running this company and previously as
an academic this is what I think is easy
about graphs. People look at them and go
well that's fantastic. I have a great
understanding of what this is. And
someone else says me too. And there
isn't necessarily a consistency in what
those two people just said. They may
have a different understanding of what
is represented.
broadly throughout my career. These are
the things that are difficult about
graphs, right? And you can say that
they're nodes connected to edges, you
can say they're distributed, you can say
they're backed up. There's a variety of
ways in which people use the data uh
that they have, the way they store it,
and the way they talk about it. And now,
as graphs have become very necessary and
consistent for things like graph rag,
for things like structured data, etc.,
more and more people are coming to this
relatively niche area previously that
even at the time wasn't necessarily
agreed upon what it was. So I do like to
define what it is we're using. So graphs
and multi- aent systems, these are the
two things that I want to define as
there's a variety of ways that people
use them.
This is how we use multi-agent systems,
right?
So now multi- aent systems are all the
way from very specifically define what
you're dealing with and chain those
together and use an LLM to glue it all
together or it is in our case break down
a complicated white collar workflow down
into a specific set of steps that I can
IO test right and each of those steps
have different requirements different
frequencies different state and that
state can be controlled often in our
case by a graph
This is why we like to use them when
we're building an application for the
legal industry. Not sure if you guys
know this, but lawyers don't really like
when things are incorrect, right? It is
basically the whole industry is make
this very specifically correct and
proper and definitely in the right
language. So when it comes to building
applications, probabilistic large
language models don't necessarily work
for that just in isolation. I need to
have a very specific control and
structure and schema for the way that we
build these systems. And I need to be
able to test and be able to pinpoint
exactly what is going right and wrong at
any point in time.
Here are some of the issues with that,
right? And we've heard about multi well
at least I've heard about multi-agent
systems a lot. I'm sure other people
have as well.
Sometimes the part in the workflow is
much more important than the other part.
Sometimes there's parts in the workflow
I don't particularly care about. Uh
there are also agents in the world.
Agents imply that these things are very
capable. But I can write a bad prompt
very easily and all of a sudden I have a
bad agent. So when it comes to what is
the agent that I trust, very few. We
spend a lot of time guardrailing as much
as we possibly can. We spend time making
sure that the memor is not just
immediate but it's episodic. We spend
time capturing the information state
over time and then pruning that state.
And again to bring it back capturing,
expanding, pruning, structuring and then
querying state for us happens in a
graphical format because the necessity
of having the structure, having the
extendability and then having the
ability to remove that extension is
really important for us. And then
finally, I'm trying not to make this too
in deep depth and too many numbers, but
95% accuracy for a single agent I think
is a tall order at this point. Maybe
people have entirely accurate agents.
I'm very happy for you. I don't have
that exactly right now. I have systems
that I can put in place like guardrails
and humans in the loop that can bring
these agents to a point that it is
accurate enough that people are willing
to use them. However, five 95% accurate
agents chained together sequentially.
That's 77% expected accuracy. That's not
that many agents in a row. If you think
about a workflow, that's five steps. And
if I'm basically saying that if each of
those five steps are 95% accurate,
already quite a hard thing to ask,
especially if there's an LLM involved,
now we're at 77% of the time it gets to
the end of that workflow in the way that
I want. That is part of probably if I
was to summarize my main problem would
be that it' be decision-m under
uncertainty throughout the process of
building these systems.
That's the background. That's that's how
we understand these systems. We use
multi- aent systems and we're naturally
skeptical. We use graphs every day and
we have a natural skepticism of exactly
how these things are stored and
structured but we use them specifically
and consistently in the way that we
like. So I am using the term agent
because everyone's using the term agent
and we build litigation agents.
Litigation is the process of well I'm
going to summarize but we work with
class action/masstor law as I said
before get everyone together they were
harmed put that harm all in place and
then sue a pharmaceutical company. Now,
we don't do any of the litigating as a
company or the suing, but we do support
the lawyers who do that. We do that in a
few different ways.
Here is one of the ways that we look at
the legal industry, right? Without
exception, everything needs to be
perfect. It needs to be accurate. Needs
to be written in the correct way, right?
There's also once you have that correct
format, creative arguments. The best
lawyers are very very very detail-
oriented and then very very creative in
the way that they can apply those
details to a case. For example, there
was an issue with uh Netflix and they
were uh capturing data from their users
as they do and they should and I'm a
Netflix user and they capture my data
and I appreciate it because they give me
the better shows that I'd like to watch.
However, there is a legal limit as to
how much information they can capture
from me, right? And you cannot surpass
that legal limit or you can but then you
can go into the process of litigation.
Now if you surpass that there needs to
be a precedent as to why someone could
say you cannot capture this much
information. And the particular
precedent I'm referring to is many years
ago Blockbuster was sued by keeping too
many details about the literal physical
DVDs that people rented. That is a
reasonably creative way to say look I
remember that Blockbuster happened and
what Netflix is doing isn't that
different. It may be in a digital
format. It may be at a larger scale. It
may be into an algorithm instead of
someone who's recommending it. However,
that is an interesting application of
what I'm doing.
So these problems then which is ne
necessary accuracy and then creativity
on top of that accuracy and then all of
that information is kept in separate
places and a lot of that creativity
comes from the latent knowledge in the
expert's head starts to come to a bit of
a four when you say well I have these
probabilistic agents that you could
argue aren't that creative right I have
these agents that most of the time do a
pretty good job and can be creative in a
way that frankly can be quite
frustrating especially to a lawyer
So this butts heads in terms of exactly
how lawyers want to deal with this
information. And again, I'm painting a
very broad brush. I am not a lawyer. My
co-founder is. If anyone is a lawyer in
the audience is offended, I do
apologize. But this is broadly what I've
seen to be accurate.
We help with legal discovery as well,
right? Like I described before, there
could be an unnamed pharmaceutical
company. A pharmaceutical company's
great, but they happen to have done some
harm, right? And it is in their best
interest to give all of the information
to the law firm and describe exactly
well not exactly describe in as many
ways as possible. Here is 500 GB of
emails that don't matter. Go nuts,
right? Figure out exactly what happened
at what point and bring up the
information. Now that is a challenge at
the moment. A lot of the time it's
manually reviewed. There are shortcuts
and processes by necessity because a lot
of these lawsuits are on a particular
timeline. It is physically impossible to
read all of the information that is
given in the discovery of the processing
of a lawsuit. However, and this is just
a generic graph I use because I'm not
allowed to use the ones that I'm
currently working on. However, if you
can take all of that information, you
can extract the information and
structure it in such a way that it is
consistent, all of a sudden that
mountain of emails becomes a lot of
information I can immediately dismiss
and a bunch of generally, you know,
genuinely useful information that I can
look at. And not just that, when it
comes to a graph, I can actually augment
the information from discovery and then
I can give that visual to the expert who
can make an immediate decision. I'm
going to loop back to the example I was
working describing before, which is the
pharmaceutical example. So again, if
ingredients are a certain concentration,
that concentration is at a problem. That
problem happened at a certain time.
There is only going to be a few people
in that graph of potentially millions of
nodes that are a problem, right? in the
same way that there are only few people
in that mountain of documents that were
a problem. However, now I've changed the
form factor such that I can specifically
hone in on what matters and not just
hone in in a datadriven way. I can hone
in in a visual way and natural language
such that the lawyer who knows exactly
what that natural language means or the
expert who knows exactly what that
natural language means can make a
decision that's data driven.
There's also a process of if we can
build this information exactly and I'm
giving the fundamentals. This is a graph
rag talk. We want to bring this graph
in. The graph I just described is not
that large. The graph I just described
has a consistent schema and the graph I
just described can be relatively easily
retrieved. I'm not going to say that
retrieval is completely solved. I am
going to say we have agents in
production right now that lawyers can in
natural language query and further
understand the lawsuit and the
individuals that they're representing.
I'll get to case research. So that was
more discovery, right? Mountain of
documents. Case research would be a lot
of people used said product and they're
complaining about it online. And this is
a lot of the value of our company and
what we do. People can complain all the
time. They can shout into the void of a
niche subreddit or they can go on
Twitter or they can be on a forum that
they're used to. They can be an IRC
where they can be wherever they want,
right? But they're using similar
language about a specific thing. And so
when it comes to traditional case
research, that information isn't really
discovered. A lot of the time it happens
through talking to another individual,
subscribing to a newsletter, etc. How do
people find the information?
So, and this is a graphic I've taken
from our website, which I promise looks
significantly better than the slides
that I make, but I tend to try and talk
to them. Uh, here is how case research
in our case for our business works and
that is we start and scrape the entire
web. Now, anyone can scrape the entire
web. It's doable. It's a technical
challenge, but it's doable. And you can
scrape it at a frequency and the
services etc. What we do is scrape the
web and then qualify the leads of that
scraping. We filter down all of the
information down to specifically what
the individuals want. We have schemas
that we work with particular law firms
and lawyers. And those schemas get us
down to just the information that they
care about. And look, maybe there is,
but right now, at least for me, there's
no such thing as a perfect case. There's
no such thing as a perfect lawsuit. It
depends on the lawyer or the partner or
the firm who's willing to take that on.
So, it is not a problem of best. It's a
problem of specific and personalized.
And that is where things like LLMs are
particularly useful at the moment.
That's where things like multi- aent
systems are fantastic. That's where
things like structured information and
graphs all of a sudden a different
lawyer can have a different multi- aent
system and a different graph that backs
up their specific way that they like to
work as opposed to having to compromise
previously on the way that everyone else
like to work or maybe hear something if
they can. And from there once we've
honed down just to the signals that they
care about the qualified signals that
are specific to them that signal can
then further generate a report and that
report can be entirely specific to the
lawyer as well. So when it comes to
report generation again a multi- aent
system that's backed up by a schema and
that schema is consistent and pruned and
that schema looks like controlled state
with a graph that can build the report
that the lawyer wants and every report
is going to be different but the
structure is going to be the same for
each lawyer and each lawyer has a
different process.
What I'm broadly describing is mass
scraping the web down to a specific
signal generated just for the lawyer.
It's entirely personalized service
that's been automated and that is the
process of what we do and this is part
of how we are able to manage and use
state and graphs and multi- aent systems
to bring the information together.
Cool. I'm going to go through I think I
have one case study um that I want to
describe just conscious of time.
This happens. Um, it's not great. No one
really wants it to. There may be
situations in which there's a bunch of
people who bought a car who really
wanted it to catch fire. We don't
necessarily deal with them. What we do
find is that there are people who are
driving their car and it starts to smoke
and then it catches fire and there's not
the behavior that they intended for to
happen, right? It was not on the
brochure when they bought it. It's not
what they want. Those people immediately
go and complain as they should, right?
They go to government website. They go
to carcomplaints.com. They're on a
specific subreddit or forum. And once we
can start to track that, which we can,
and once we can start to scrape and then
structure and then schematize and then
analyze, we can start to basically build
a density of complaints for a specific
vehicle, for a specific year, for a
specific problem. And that density is a
combination of how many complaints
multiplied by the velocity of
complaints. So a certain amount per
month over a, you know, number of
months. All of a sudden, we get to the
point where we're finding these leads
particularly early. And now as we're
building models, we're starting to find
these leads early and earlier and that
we don't necessarily need the velocity
straight away. We can start to figure
out what are the previous lawsuits which
were all public and very well documented
and exactly what happened in that
process. And so for a large law firm,
maybe eight or nine months post people
starting to complain, they can take that
lawsuit on if they want to. Uh for us,
we can find it uh within about 15
minutes and then generally it takes
probably a month for you to be confident
that this is the signal that you want.
And so we can find things significantly
earlier that process again scraping the
web filtering down producing the
specific report. This is an example that
we did. And again uh we deal with what
the lawyers want. So this lawyer again
he made the case that uh people's cars
are catching fires. They don't really
want them to. Those are the cases that
he would like to take on. It's of a
certain amount of money. It's of a
certain make and model. It's in a
certain jurisdiction etc. Those specific
filters that schema can be applied
throughout the entire process. That's
basically the graph. Each of these
lawyers have a specific graph that they
want. And not just that they can filter
and feedback that information. So it's
not just a static graph. I mean the
benefit of a graph structure at least
well one of the benefits of a graph
structure I should say is that it's an
extensible schema and that I can update
and I can query across and I can
understand that information. And so uh
while we are dealing with rag I would
say we have less of a chat rag
interface. While the lawyers definitely
do appreciate that, a lot of what we
have when it comes to rag or retrieval
augmented generation would be generating
these reports because as much as a
lawyer does want an answer, what they
also want is the form factor that
they're used to. And so all of these
graphs are consistently made and built
each day and then some subgraph from
that broader monolithic structure is
then brought in and composed into a
report that a lawyer can action.
Kind of what's what's next and I'll talk
about the future a little bit. I mean
what I described is kind of what we're
doing. But this is what we're doing.
Final lawsuits early and compensate harm
and then people can have that
information if they want to. Um we're
able to do this entirely technically.
We're able to scrape the web structure
etc. We're able to iteratively build up
a schema as we want to. Uh this is not
just a genai problem and I think this is
an important thing that I've seen around
this conference and people may be seeing
is that genai is not better than machine
learning. uh and LLMs are not you know
better than traditional ML systems but
there are situations in which one is
fantastic and one is not. If you look at
multi-agent systems and again I was
previously an academic in multi- aent
systems and no one ever listened to me
so this is a bizarre situation but that
when you used to structure the multi-
aent systems together somewhere along
that workflow you would have to stop or
say this is not doable because I cannot
plug these two bits of information
together it's too probabilistic or it's
too random or it's too inconsistent or
the way to describe it is not a binary
feature right it is I really just want
to kind of type what I want right now
with LLMs you can but it's very much for
us not an LLM filtered system. It's an
ML filtered system that LLMs have
allowed us to pipe together such that
you can actually provide value
completely end to end which I think was
previously not doable. And for us,
again, we've been using graphs for a
long time. For us, the ability to
iteratively build that graph, prune that
graph, and every single report gets
better because we're able to manage the
state is why people like working with us
because we can consistently follow and
track exactly what they want
specifically.
Cool. I think I'm just about at time.
Kind of got in early, but that's been
the talk specifically around I'm happy
to talk to anyone about the specifics um
graph rag etc. multi- aent systems, but
that's how we use the process. Thank you
very much.