TLDR
Helped a UK travel company build an AI agent for end-to-end trip planning. Talked them out of a trendy swarm architecture, shipped with a simpler approach instead. Saved ~1 month of engineering time. Set up evals so the team can iterate systematically.
The Problem
The client wanted to build an AI agent that could plan entire trips across flights, hotels, and activities. Users would describe what they wanted, and the agent would prepare the trip with a human-in-the-loop flow for booking.
Their engineers wanted to build a swarm architecture. It was the hot thing at the time. Multiple specialized agents coordinating with each other, very sophisticated.
Complex architectures have too much surface area. They’re hard to build, hard to debug, and even harder to iterate on. When something breaks, you don’t know where to look.
The Solution
Start simple. Add complexity only when you need it.
I proposed starting with a single ReAct agent with tool calls and adding complexity only when eval failures justified it.
We shipped fast and covered most of their business use cases with this approach.
Only when the simpler design hit real limits did we add a lightweight supervisor. It was enough coordination for the edge cases without committing to a full swarm.
What I Built
Phase 1: Simple Agent
Single agent with tool calls. Connected to real services: flights, hotels, activities. Human-in-the-loop workflow for booking confirmation.
This got us to a working product fast.
Phase 2: Supervisor (when needed)
When the single agent started hitting its limits, we added a lightweight supervisor. We did this after we had evidence it was needed.
Phase 3: Evals
This is the part that pays off long after I’m gone.
I integrated an eval framework into their workflow. Now:
- Bad prompt changes get caught before shipping
- Engineers can test new models quickly when they come out
- They can measure cost/latency/accuracy trade-offs systematically
Knowledge Transfer
Jumped into their codebase, shipped it myself, then taught their engineers the patterns. The goal was always to set them up to continue without me.
Results
- ~1 month saved by avoiding premature swarm architecture
- Evals: repeatable quality checks before prompt or model changes ship
- Team can now independently test new models and iterate on agent performance
- Stayed on for a few months of advisory to help when they got stuck
What I Learned
Everyone wants to build the sophisticated thing first. You don’t know what complexity you actually need until you hit real problems. Ship the simple version, then evolve.
Most teams treat evals as a nice-to-have. Baking them into the workflow means bad changes get caught automatically and new models get tested systematically. That compounds over time.
Related
- AI Consulting: More about my consulting work