One-time engagement
Agent Production Readiness Audit
A three to four week review of your live AI agent, tested on your real use cases and scored on six gates, that ends in a scorecard, a cost-per-resolved-task baseline and a ranked 90‑day roadmap.
Find out what breaks with real customers before they do.
- 3 to 4 weeks
- From $2,900
- Scorecard, cost baseline, 90‑day roadmap
Is this for you?
Your agent is live, or goes live this quarter
Real customers are on the other end, and the first incidents will be yours to explain.
A buyer, a board or an investor wants evidence
Someone needs proof that it holds, in writing, from outside the team that built it.
Usage is about to grow
You want to know what breaks first at ten times today’s traffic, and what it costs to run.
Not this? The architecture review covers smaller systems; the AI security review covers only security.
What you get
- 01
The scorecard
Six gates, each scored on your system, with the evidence behind the score and what to change.
- 02
The cost baseline
What one resolved task costs you today, by model, by step and by retry, built from your traces and your bill.
- 03
The 90-day roadmap
Every fix ranked by what it is worth and what it takes, written for your team to act on without me.
Illustrative pages. Yours carry your system’s scores, numbers and fixes.
Ranked 90-day roadmap
- 1EffortWorth
- 2EffortWorth
- 3EffortWorth
- 4EffortWorth
- 5EffortWorth
Cost per resolved task
| Model | Step | Retries | Per task |
|---|---|---|---|
Production readiness scorecard
Six gates, scored on your system
- Reliability
- Eval coverage
- Security and guardrails
- Observability
- Cost per resolved task
- Incident readiness
What I check
Your real use cases run through all six, many times over. Each gate gets a score, the evidence behind it and what to change.
01Reliability
I run your real tasks many times over and measure how often the agent gets them right every single time, not just once.
I measurepass^k over repeated runs of your real use cases
02Eval coverage
I check how much of what your customers actually do is tested before a release, and leave your team a test set that covers the gaps.
I measureThe share of real traffic the evals cover, and the gaps
03Security and guardrails
I test what a customer, a document or a web page can make the agent do, from leaking another customer’s data to misusing a tool.
I measureInjection, data crossing and tool scope, each as pass or fail
The AI security review goes deeper04Observability
I check that every step is recorded, so when an answer goes wrong anyone on your team can see why and replay it.
I measureWhether every step is traced and a bad answer can be replayed
05Cost per resolved task
I work out what one finished task really costs you, by model, step and retry, from your own traces and your bill.
I measureDollars per task, by model, step and retry
06Incident readiness
I check what happens when a model provider slows down, a tool fails or an answer is wrong: the fallbacks, the rollback and who gets told.
I measureFallbacks, rollback and alerts, checked on one real failure
3 to 4 weeks, written report and readout
The four weeks
- Week 1
Read the system
Access in place, the code and architecture read end to end, your real use cases collected with the people who know them.
- Week 2
Test it
The use cases run against the agent many times over, the traces and the bill read, each gate scored with its evidence.
- Week 3
Rank the fixes
Every finding ranked by what it costs you and what fixing it is worth; the cost baseline built, the roadmap drafted.
- Week 4
Hand it over
A readout with your team, the questions answered, the roadmap agreed. A smaller system finishes at the end of week 3.
The next step
Start with a
20-minute call.
Tell me what the agent does and who depends on it. You leave knowing whether the audit fits, how long it would take and what it would cost.
Bring
- What the agent does, in a sentence
- Who depends on it: customers, a buyer or a board
- Where it worries you most
Why me
Sixteen years of production systems, the last six in AI with customers on the other end. I was Fractional CTO at Devnagri AI, a localisation platform working across 22 languages, and I am Chief AI Architect at TWINii, where real-time AI twins of real experts went live with memory, guardrails and cloned voice. The six gates are the places I have seen those systems tested, not a checklist from a book.
The track record“Mandeep didn't just put in a patch, he helped redesign the system architecture to handle 5x the load without too much of extra server hardware.”
Jaspreet Singh OberoiHead of Engineering, DevnagriAsked on most first calls
The questions.
How long does the audit take?
Three to four weeks from the day access is in place. A single agent with a handful of tools is usually three; several agents, many tools or more than one language usually four.
What access do you need?
Read access to the code and architecture docs, a sample of staging or production traces, the billing data for the models you use and time with whoever owns the system. Nothing needs to be installed in production.
We do not have evals yet. Is that a problem?
No. Most teams I meet do not. The audit builds the first set from your real use cases, and it stays with your team.
Do you sign NDAs?
Yes, yours or mine, before I see anything.
What if you find nothing serious?
Then you have a scorecard and a cost baseline that say so, in writing, which is what an enterprise buyer, a board or an investor is asking for.
What does it cost?
From $2,900. The price moves with the size of the system, so it is fixed on the first call, in writing, before anything starts.
What happens after the audit?
The roadmap is written for your team to act on without me. If you want me on it, I can join part-time as your fractional CTO or AI architect and carry the roadmap with you.
Which stacks do you work with?
I have shipped on Node.js and TypeScript, Python, PostgreSQL, MongoDB, Redis, Google Cloud, AWS, Azure and Kubernetes, and I work across retrieval, agent orchestration, evals, guardrails and real-time voice. If yours is unusual, ask on the call.