> ./blog / microservices-ai-agents
I Hated Our Microservices for Years. Agents Changed My Mind
I see lots of debate about AI compatible architecture and monorepos but I rarely see anybody mention microservices. I believe it’s an ideal architecture for agents and I’m as shocked as anyone by that, as I’ve hated ours for years.
This is how I went from hating the architecture I inherited to thinking it’s ideal for the agentic development era.
Inheriting a mess
When I joined my startup as founding CTO, I inherited four or five services and a team of one: me. The team that built the MVP had gone fully pilled on microservices and serverless, and it was all dropped in my lap.
The boundaries weren’t really boundaries. Services made direct calls into each other, and they all spoke to the same database. Yes, it was that bad. Whenever I needed to work on something end to end, I was running every service locally.
I basically hated it. For a team of one, it was all of the overhead of a distributed system and none of the payoff.
The first two benefits
Now a team of two, we started refactoring away from the poorly built Node services, and that’s when the first benefit showed up. I wasn’t tied to their tech stack. So we built the first .NET service in the system and ran it alongside everything else.
Then, when it came to adding the next service, we hit an inflection point. We could fold everything into the .NET service and end up with a monolith, or we could lean into the microservices of it all.
We leaned in. Over time we replaced everything we’d inherited, one service at a time, with no loss of service. That was the second benefit.
By the end, the boundaries were properly defined and every service had its own database. It finally looked like the architecture it had been pretending to be all along.
Then agents turned up
This is where all that work paid off.
Microservices were always a team-scaling architecture. The idea is that separate teams own separate services and ship without treading on each other’s toes. The catch is that most teams were never big enough to need that, and we certainly weren’t. Two people don’t need team scaling.
Agents changed that. Each agent is effectively a team. With a team of two, we can have an agent working on each part of the system at once, without ever having to use a worktree.
Each agent only needs to know its own narrow context. There’s no huge codebase to hold in context. It can change whatever it wants internally with no risk to the external interface. It can add new functionality to its service and keep it off the API gateway until it’s ready.
Better being oblivious
The way we run them is simple. Each agent can see its service’s outbound contract, and that’s all it needs. They’re better being oblivious to everything else.
We keep each change scoped to what that service needs to do. When a big change touches several services, we split it into a slice per service rather than treating it as one big thing. Each slice is something one agent can own, and the contracts are the only thing that has to line up.
It’s the same discipline the architecture always asked for. It just took agents for us to really feel the benefit of it.
What it does to token spend
I don’t have hard numbers, so I won’t pretend to. What I can tell you is this. We ship many times a day to a mature codebase. We use Opus exclusively. We’re both on the lowest Claude Team plan, at $20 a seat a month, and we rarely run out of tokens in a five-hour session.
I’m actually waiting for the day we upgrade to the $100 plans, but it never comes.
I put that down to the narrow context. An agent that only has to understand one service doesn’t burn tokens reading the rest of the system.
The overhead hasn’t gone anywhere
I’m not going to pretend microservices got easy. All the usual overhead is still there:
- Multiple representations of the same state across services.
- Publishing events to keep other services up to date.
- End-to-end debugging.
This is how we live with it. We have a development environment with everything in it, and a local API gateway that can be repointed at a local instance of a specific service. So I can run the one service I’m changing and still exercise it through the gateway.
Mostly, though, it’s continuous delivery and extensive TDD. That’s what gives us the confidence to ship without an end-to-end test locally, and to test in production.
Publishing events can still be annoying. The key there is chunky, coarse-grained events rather than lots of tiny ones.
So are monorepos wrong for agents?
I don’t think it’s that simple, and I can see why people like them. Everything in one place has real advantages, especially for changes that cut across the whole system, and plenty of teams make monorepos work well with agents.
My point is narrower. If you’ve already got services with clear boundaries, agents might be the thing that finally makes them worth it. The architecture I spent years lamenting was built for a team size we didn’t have. With agents, we’ve got it.
Of course, you can also keep microservices in a monorepo. However, I’d question why you would want your agents grepping all of that context when you have well-defined, independently deployable services.
So maybe you’re sat on a microservices setup you’ve always resented. Perhaps somebody drank the AWS Kool-Aid and you inherited the product of that. If so, give it a second look before you fold it all back into a monolith.