How AI is changing Software Engineering: A Conversation with Gergely Orosz, @pragmaticengineer

Channel: aiDotEngineer

Published at: 2026-04-21

YouTube video id: CS5Cmz5FssI

Source: https://www.youtube.com/watch?v=CS5Cmz5FssI

Sure
game.
>> All right. I going to assume most of you
uh show of hands who subscribes to
Pragmatic Engineer. Oh my god.
>> Wow.
uh he is uh he needs no introduction.
Then let's get right into it. Um
what is token maxing and should everyone
here be doing it?
>> So I I heard about token maxing a week
ago or like week and a half ago first
and you know some people have been doing
it for longer and I tweeted about it I
think three days ago saying oh there's
this token maxing and again you see it
on social media and my DMs were blowing
up from from people at large companies.
I don't want to name names but like you
know Meta, Microsoft
uh some so some some other ones as well
like uh the likes of and and and so so
many more and the story is a little bit
different every at every company on why
people are doing it and whether they
like it or whether they think it's good.
But there's a few a few common themes.
One is token output at these larger
companies is measured in in some way.
There's like either a leaderboard or
there's a way to look up your your
peers. Salesforce, for example, you can
check the spend the the money spent that
every every person at the company did.
You can like search in a tool that
someone built and it shows how many
dollars they spent on on AI related
tokens. And you know, first there's this
number, then there's this uncertainty on
in the tech industry, right? We're kind
of hearing layoffs, like massive cuts at
the likes of block. And I mean there
like no matter how much tokens people
spend they were let go independent of
this but people start to think like does
is it part of performance evaluations or
promotions or all that and the answer is
kind of. So inside of meta I talk with
managers and in the performance
evaluation they have this data point
which is one of many data points right
the same way as as like diffs or impact
or or code reviews of how helpful this
person is but they do just like with any
data point they sometimes pull it in and
use it. So typically in just like any
data point it can be weaponized. So like
a low performer with low impact and a
low token count clearly not even trying.
So, and a high performer with high
impact and high token count. Clearly,
that's innovating and this must be doing
good. So, inside of these companies
specifically, I talked with a lot of
people at at Meta. And again, this is
not representative 100% of Meta, but
they had this leaderboard where people
showed up and they have like massive
amounts of tokens and a lot of engineers
got just scared, worried, so they
started to token max to try to generate
tokens. stories that I've heard first or
well secondhand from these people who
who who told me firsthand is for example
instead of reading the documentation I
will ask the agent to summarize it for
me and ask questions even though it
doesn't do a good job answering it but
my token count goes up people just want
to not be in the bottom 25% or bottom
50% for token count where these things
are measured inside of Microsoft again
there's a leaderboard and I'm talking
with people they're like it's ridiculous
like how some people are just running
autonomous agents to build junk honestly
for the sake of having that number go up
and and sometimes it gets ridiculous
because like inside of Meta they had
this leaderboard they got rid of it
after an article came out and it looked
amaz
like just just like closed it down. that
people are still token maxing by the way
because there's this this thinking that
it might have gone but you know we're
engineers and don't forget these are
highp paying jobs right that like you
don't really want to lose a job over
something stupid as like you didn't have
INF token count and that's how it feels
but inside Salesforce there's a target
of minimum spend per month like I think
it's like $175 between the things so
like people are like again you kind of
like you know beginning of the month
like just token max to get there so it's
it's it's weird and it started as a joke
earlier like a few months ago token
maxing was really just people like going
crazy and enjoying this thing and
building cool stuff. But it's kind of
turned into in a lot of companies I
think it's just a culturally weird
thing. So it's a weird time to be in cuz
I remember lines of code used to be when
when early uh developer productivity
tools came out like velocity and
pluralite flow. They kind of measured
lines of code and and number of QPRs and
we know that was stupid and people kind
of optimized for that at companies that
did it. But it's it's almost like what
now it's the top running companies like
Meta and Microsoft who are incentivizing
people just to do just stupid stuff
honestly.
>> Yeah, those are wild stories. And one of
the things you're clapping for that
deserves another full conversation. Uh
one of the things I like about talking
with you and subscribing to your
newsletter is that you basically kind of
anonymize all these stories from from
real incidents and real examples. Um why
is it that uh is is it still worth it
right with all the flaws uh you know
when you have good heart's law like what
whatever gets measured gets uh sort of
abused with all the flaws is it still
worth it you know is is is AI basically
still making us faster overall like the
cost of token maxing is still with all
these like really ridiculous examples is
it still net worth it
>> yeah so don't forget like the reason
token maxing is probably a thing is like
let's just go back to six months ago
where
I I I was at a I was at a CTO like
dinner conference whatever like a bunch
of CTO's gather CTO level people this
this was in Amsterdam and we had like
like a bunch of people and there we were
talking and and one of the CTO's like
the the the Amazon of the Netherlands
there there's a e-commerce company was
saying like hey like everyone like I
have a problem like engineers on my team
are really skeptical of AI and they're
not really using it. The AI tools, don't
forget this was before Opus 4.5 and
those models were were out. They were
not as as productive. We had uh we we
already had a cursor and and the like
and they subscribed. They're like
they're just not using it that much on
existing code bases, right? And and next
to them uh the head of the Dutch
National Bank said like, "Oh, we don't
have that problem. Our engineers are
using it because our our mission is to
regulate this thing. So, we need to
understand it." And they're kind of
motivated. And there was this time where
experienced engineers were kind of
holding off because if you had an
existing codebase and use AI cursor
whatever on it was mildly useful if that
even and these engineers were like why
should I use a tool if it doesn't help
me refactor it doesn't find the bug it
doesn't do what I need to do and
leadership saw they're not really using
it and they kept hearing you know the
likes of Antrophic for example was
already saying how they're writing a lot
of their code with with cloth code uh
and it just keeps increasing and
andropics, you know, like revenue is
going up like this. So those leaders are
kind of they might be confusing
correlation and and and you know, like
which one comes first, but they're like,
well, we should be using it more because
probably good things will happen and
thus bad things will happen if we don't
use it. So the whole targeting and
measuring things, it actually came from
leadership wanting, we want our
engineers to use faking AI. I don't care
what it is. And it it was a bit of a
push like we know this is bad but it's
it's better than them using it. Best
example is Coinbase where uh Brian
Armstrong the CEO just like fired an
engineer or he sent an email saying
everyone like needs to get on board and
use AI tools and whoever doesn't use it
in a week I'll have a conversation with
them and then I think a week later on
Saturday he fired an engineer and you
know like this again high paying job
like we're talking base salary like
three 400k,000
per per year uh and then both equity and
everything on top of it like they got
the message everyone just started to
just you know like use it and you back
to your question. So on on one there
there's a push and look I feel it's a
little bit like this is going to be
controversial but have you ever wor
wonder wondered why big tech loves to do
lead code style interviews algorithmical
interviews which have nothing to do with
the job and and we know it's the case
and there's a lot of criticism for this
and they've been doing this since since
like 20 years but here's the thing it
selects for a specific type of person.
It selects for the person who's smart
and willing to put up with absolute
[ __ ] to get the job.
And this person, you know, they will
study two months pre AI, two months or
three months of lead code, which again
makes no sense on the job, but you do
it. You get in there and this person
will be putting to put up with [ __ ]
that makes absolute no sense to keep the
job. So token maxing happens at large
companies and people are putting up with
this BS. And look, a lot of them are
smart and they will make the most of it.
some of them will build cool stuff. Um
it's it's the reality I think of big
tech. So we're in this weird place where
big tech is a bit weirder than startups
where you know no one cares about
tokenaxing. They care about like just
building stuff and you know use whatever
makes sense. Don't people will care
about the cost.
>> Yeah.
>> But going back to your question like
like you know like is is it making us
productive as as a whole like
individually it's it certainly is and as
teams we're kind of like a bit question
mark because we should be moving faster
and there are a few companies that do.
Entrophic is a good example, but a bunch
of companies are like not it's it's it
seems it's hard to retrofit all this AI
into like the way we have been working.
>> Yeah. Uh one of my favorite studies from
last year was the meter study where they
uh did a blind test of uh people and
their expectations of productivity,
right? And basically the the end result
was they felt 20% more productive, but
their demonstrated results was actually
they were 20% less productive on
average. Yes. But that that study was
very interesting because they
>> it was very small sample size.
>> It was 30 people and there was one
outlier uh who actually was way more
>> Anthony we we interviewed him on the
pod. Yeah.
>> Yeah. Yeah. So he was the one productive
AI engineer
but anyway so uh actually my theory is
that uh something that I've seen on my
team is that I've been enabling coding
agents for the rest of my team who are
non techchnical right and uh you as the
engineer may not be more much that much
more productive because and you can be
more productive if you uh attend AIE but
uh if you actually enable your
non-coding uh your your non-coding co
collaborators to code actually they are
more productive because they don't have
to wait for you right and that's that
like unlock of like oh suddenly you have
serverless developers basically uh and I
think I think that's that organizational
coding thing is different than studying
pull request level productivity for the
individual developer
>> yeah and and the thing that still I
still remember to this date I I talked
with Simon Willis I think in 2024 so two
years after Chad GPT came out and he was
Simon Wilson top commenter on hacker
news or he's he's
>> that's his that's not his title man top
commenter on hacker What the [ __ ]
>> No, he's
>> creative, Django, top blogger. Yeah. Uh,
prompt injections. Uh, yeah.
>> Yeah. He's actually not talk. I'm sure
he's the most submitted block cuz he
blocks so much like like and he's
>> but he told me back then he said like
this thing AI is is just so hard to to
get good at. He's like there's no
manual. And he's like, I've been doing
it back then for two years and I'm still
I'm still figuring out what works and
what doesn't. I keep changing my
workflows. And I think that's something
that is a bit hard for us. Two things
about AI that for any of us engineers is
hard to understand. One is it just takes
a long time to get good at it and you
need to keep doing it. And the second
thing is understanding the theory will
not make you better at using the tools
which is an absolute mind [ __ ] honestly
because we're so used to you know you
understand how the compiler works, how
assembly works. Okay, you will now be
more efficient if you want to write
low-level code because you know how it
works. But what with these things I mean
you can of course it's helpful to
understand how how the the architecture
underlying works attention the different
the the different probability sets etc
etc but it will not help you get a sense
for how you can use it and then once you
figure out how you can be more
productive if you're if you're inside of
a team again it kind of breaks and you
have to relearn again but but the more
effort you put into it it like it's
clear that it's it's working it's
helpful and I think it it's the teams
I'm seeing and getting more value out of
it. Low ego, open to learning, open to
leaving your priors behind. The word
priors I have not used forever and I
feel we're in this stage where like just
just leave your priors behind. Just have
an open mind like don't leave your
experience behind but you know be open
to it.
>> Yeah. Zooming out a little bit. How is
the role of the software engineer
changing?
>> I think it's always this was always
coming but AI is just just speeding it
up. uh even before AI a few
it's interesting I see like startups in
many ways venture funded startups are
kind of front running what the industry
will be catching up because venture
funed startups are about fast growth um
doing
mo moving fast with smaller teams
because smaller teams mean smaller comps
even preai so a lot a lot of these
venture funed startups start to expect a
lot wider range of roles from engineers
for example devops as a whole inside VC
funded companies from the mid210s every
engineer was kind of like responsible
for the code they deployed but like more
traditional companies they had more
money more sorry more less pressure they
kind of have dedicated devops teams and
some of those things so in in the
industry like the software engineer is
now becoming like the kind of the tester
role has collapsed into software
engineer we most companies don't have
dedicated testers very very few do
devops collapse into here uh and now
we're starting to have the product role
also starting to come so a lot of
companies even like in 2022 before AI
starts to hire for product engineers
that's happening faster and I think the
the last push that AI is doing is even
for early career engineers there's a lot
more seniority expected or or senior
like things planning about things
knowing about the business so I I I
think the role is expectations are are
higher teams are also getting smaller
everywhere I talked with someone at John
Deere 200 person uh 200 year old company
sorry uh you know like they do tractors
and and all all that stuff and and
inside of that company, one of their
their VP of engineering was telling me
how they're actually seeing that their
two pizza teams are now just one pizza
teams inside of that company. It's the
reality partially because of these
tools.
>> So, my joke used to be I am a one pizza
team because I eat a lot of pizza, but
uh depends how much pizza you eat.
>> Uh there's so I'm sorry to interrupt. I
don't know if I cut you off in some
critical point. Uh there's a comment
saying I've heard it twice even among
this audience where a lot of people are
saying that oh uh you're no longer an
engineer everyone's an engineering
manager now and you've been an
engineering manager and I wonder if you
agree with that or if you have a
different take you know because
basically you're the the the common
analogy is that you're no longer a
software engineer you're just managing
engineering agents right yeah if you've
been a manager before that is an
absolute [ __ ]
so so here here's the thing the like
Yes, you are a manager without all the
things that no one wants to become a
manager for the the when you become an
engineering manager. Hands up if you are
or have been an engineering manager,
right? Hands up if you actually if
you've not been and you want to be one
>> about 15 20%.
>> All right, you come and talk to me
afterwards. I I'll tell there's a hand
up there. I'll talk you out of it. So,
so what you think you become an
engineering manager to like help
people's career, maybe have higher
salary, higher impact, all you know
there can be a lot of dynamics but the
reality is is is you you become more
removed from the product and you have to
deal with people problems and the thing
with with agents is you don't have to
deal with people drama, people problems,
conflict between your team. I mean
unless the next generation of agents
starts to fight with each other. I think
that'll be something but you actually
you you do have to orchestrate but it's
more like a tech lead role or or or
experienced engineer where where you're
like mentoring uh mentoring engineers
but you don't have the people
management. You don't need to worry
about the personal problems. So it's
actually a lot more kind of empowering.
And I was talking with uh the podcast
was was just out yesterday with with DHH
uh creator of Ruby on Rails who said,
you know, people told him like, okay,
it's it's like managing things and he's
not excited about managing agents, but
it feels it's more like a mech suit
where you have like you can do seven
things at once, you can do a lot faster
and you're in control and that's more
what it feels like. So there's
orchestration, yes, but it's very
different to management. And also the
the really really bad thing or honestly
shitty thing about management if if you
make it into management which makes it
hard also rewarding later when you you
tell yourself at least this thing is you
start a project with all these people
under you you know congratulations
you've got 10 people wonderful and you
start a project and in 6 months you will
see some results of the decision that
you made with agents it's just so much
faster so the the feedback loop is
faster so I I think it's it's not much
of it except for the orchestration and
and and for that everyone's going to
have their own flavor. Some people will
will have the tendency to like run
multiple agents and they're good at this
or we good at it. Some people just do
like two agents. Michelle Hashimoto, I
interviewed him. He has two agents. He
always has one agent running. No, he has
one background agent that he doesn't.
That's it. He's like two is enough for
me. Great.
>> Yeah. Yeah. Uh we're figuring out the
patterns. Um uh I want to hit you on
large tech infra.
uh this is something that I think both
of us are very excited by by uh good
infra which is a very niche uh interest
what are you seeing
>> it's wild to see how much of the so I
said that from externally a lot of
companies a lot of big tech companies
especially the ones are spending a bunch
on AI and have platforms and all that
you're not seeing too much like more
come out like Uber is a good example I'm
not seeing too many more features come
out of Uber or new products launcher and
they're like but what's going on they
are really investing in AI but when you
look inside there's a whole lot of buzz
they are rebuilding their complete IM
infra you know they're and I'm not
talking about they're buying cursor or
or cloud code or all that they're doing
that as well but they're completely
they're building their own own custom
background coding agents that is
integrated into their monor repo they
are are having uh their own MCP gateway
that is is now integrated into service
discovery their on call tooling is being
retoled their internal code review
system is like like categorizing based
on risk. They are like and Uber is one
example but all everyone else Airbnb
intercom meta Microsoft even midsize
companies are just building so much
internal improp and I was asking to
myself like why on one end this feels
like such a waste but when I worked at
Uber for four years I realized they
spend so much on on internal platform
there's two reasons one is honestly it's
a it's a lowrisk way to get good with AI
uh to be hands-on and these companies
want to be hands-on but maybe you
shouldn't start with shipping AI
features no one wants into your
codebase. Second of all, because these
these companies have such so much code
that never fit in a context window, by
building custom solutions and just basic
basic wagons, that kind of stuff, they
will have better results than
off-the-shelf vendors. So, they already
have a win. And number three, honestly,
is anything that has AI in it gets
funded. So, there's this joke of if
you're in the developer platform team
and you're asking for more headcount,
like good luck with that. Oh, developer
platform. Oh, but say that you want to
get two extra headc count for agent
experience. Done. H. So, so there's that
part as well. But, but all
>> agent experience is just a CLI
>> pretty much. But all these come inside
there's so much buzz and so much work.
Everyone's building their own custom
system. So, I'm kind of wondering how
long this will take, but I think for
next year this is going to happen. So,
if you either have friends or if you're
work if you're working at a company,
you'll see. But talk with with friends
at other large companies and you will
probably see you are all building the
same thing. If you're at a large company
and you're not already building an MCP
gateway, what are you even doing?
>> Yeah. Um, actually a lot of these topics
are exactly the things I cured for
tomorrow. Uh, it's just fantastic to
have you as the closing keynote for
today because uh it's it's like a
appetizer for tomorrow. We have talks
about MCP gateway and all these sort of
AI architecture and infra things and I
do think like uh infra like
taking AI infra seriously as a company
is uh very mis not that well un
understood and right now you just kind
of learn by example from people because
there's not really like a textbook or
anything like about it. So the way I
think about this because again from if
you just kind of step out and we love to
criticize big tech of how they're
wasting money here and there and by the
way we love to criticize Google and I'm
kind of thinking to myself like hang on
what if Google ex actually executed well
like do we want that and you know they
would kill all the startups but but what
they're doing makes makes sense and
Shopify is an example where I'm like huh
I'm starting to get why it makes sense
to do all this stuff. So Shopify in 2021
they were the first company to have
access to a GitHub copilot. What
happened is the the head of engineering
fartoir heard about GitHub copilot being
developed internally inside of GitHub
and he pinged Thomas Dunca the CEO of
GitHub at the time and said hey Thomas I
heard you guys are doing C-pilot and
he's like yeah we are it's internal.
He's like I I'd like to get access to
it. He's like yeah but it's not for
sale. He's like no no no you don't
understand. I I didn't ask if it's for
sale. we would like to roll it out to
all of Shopify and in return we will
give you feedback for 3,000 people for
you know as honest feedback all the time
and so they got it a year before it was
out anywhere and they incurred a lot of
churn. It wasn't that great initially
and and they went through all of this
stuff and then Shopify was the first
company to on board to like a bunch of
other tools and they gave unlimited
budget and they're spending so much time
ironing out bugs. But the reason they're
doing it, this is what like made me
click is they are trading off churn and
expense and spending a lot more money to
be at the forefront of this. They are a
few months ahead or six months ahead of
their competition and for them it's
worth it. It's not worth it for anyone
else, right? If you're if you're at a
company where your business is like
something something physical and you
don't care like yeah just just wait out
it it'll come. But for a lot of us in
the tech industry this turn is worth it.
Plus what Farhan told me is like because
he actually told me he's kind of worried
about the cost now. But he was like look
like it's still worth it because if it
would look silly if I said you cannot
have these tools how would I hire the
best?
>> So it's it's innovation recruitment and
it kind of makes sense when you think
about it. And the weird thing everyone
is doing it at the same time. So it
looks silly but it it's rational.
>> Uh my next podcast is with Mikuel Parkin
the CTO of Shopify and uh the sheer
amount of machine learning that they do
and infra that they set up for their
customers makes me want to be a
customer. You know that's that's like
the best uh endorsement I can give. Um
I'm going to get meta a little bit and
talk about pragmatic engineer. Uh you
and I kind of startedish in COVID. Uh
you just left Uber. Uh how has it been
growing? What what are the main stats
that you're proud of that uh you'd like
to share with the world? Yeah. So I I
started pragmatic engineer I I I a joke
that if it wasn't for co I I would
probably never have started the this
thing because what happened with co is
uh Uber had layoffs and most of the tech
industry was doing great but Uber was
not and my team uh was hit by layoffs
and then we we had to disperse the
remaining people at other teams because
our mission no longer made sense and it
was just like a the morale was low my
morale was low so I was like let me take
a break. I wanted to write some books.
Swix was writing his book the the coding
career.
>> Yeah some of you have read it. I've met
some of you.
>> Yeah. and and that that's how we met
there and then uh my plan was to write a
book and then start start up some
startup something something platform
engineer control c control V from what
Uber was doing inside and that's
actually almost all Uber su Uber
startups it's it's amazing temporal is
is is from there
>> if I by the way if I did not start AI
engineer I would have started platform
engineer
>> that that would have been the industry
conference
>> yeah love it and then I start I started
the pragmatic engineer uh a year after I
left Uber It was just an experiment. Um,
I figured no one Substack was taking
off. No one was writing about software
engineering in-depth and I just acted
all confident saying pretended that I I
knew what I was doing. The first article
was about Uber's platform and program
split that no one had written about
publicly before and it's a it's a free
article. You can you can now check it
out. Uh, and it was like when you feel
product market fit, that's what I felt
almost immediately. The first week
before I published anything, just a
confident Twitter post, I had 100 people
pay upfront $100 for the whole year,
which I was like, whoa, I have published
anything. In six weeks, I was at a,000
people paying for this thing that didn't
exist before, which was my old Uber base
salary back back in Amsterdam. And it
just kept going up. So like I I figured
like when you find product market fit,
this is like outside of like there's
this rule like if you find product
market fit, just keep doing what you're
doing. So for me, I just kept writing
that one article. I got all these
interview requests, collaborations,
podcast. I just said no to all of them
because I knew the most important thing
was to do what makes it successful,
which is that one article. And later it
turned into two articles. And for two
years, this is all I did, just two
articles. And after two years, I looked
up and I was like, huh, like this is
actually working. People like doing it.
I like doing it. There's a future in
that. And that's when I decided I
actually want to turn this into a
business that I don't burn out because
for two years every vacation I went to I
was working 50 60 hours. I was always
thinking I was writing I I couldn't
really let go. So I started to grow the
team a little bit. Uh I I Ellen Bird the
first secondary researcher Ellen she's
ex x ex uh
>> she's here right?
>> Ellen's not here. Um Jessica is who who
just joined uh later.
>> Yeah.
>> And then uh so now it was two of us. Uh,
and I started a podcast year and a half
ago because I talked with so many
people. I figured it was a bit of a
shame to to not have it. So, the
primatic engineer became the number one
paid technology newsletter about four
months after starting. It stayed there
for three years. Now, semi analysis has
>> Dylan versus uh you guys. Um, yeah. No,
congrats on your success. Uh I think you
you're also a leading tech voice in
Europe which I think you're sort of
proudly sort of uh upholding that over
here which I would really wanted to
feature. Thank you for your support for
AIE. And uh everyone thank you Good.
Awesome.
Thanks, man.