Agent projects are usually stopped for reasons the model did not cause. The fatal choices are made in the first weeks: no test set, no cost limit, no list of permitted actions, no route to a person, and a plan that treats the connections to core systems as a detail. Each one has a small, buildable fix.
Gartner forecast in June 2025 that over 40 per cent of agentic AI projects would be dropped by the end of 2027. Reuters reported the reasons as escalating costs, unclear business value and inadequate risk controls. None of the three is a statement about model capability.
Five missing controls and what they look like in month three
| Missing control | What the review meeting hears | What to build instead |
|---|---|---|
| A test set | "It seemed better last week, but we cannot show it" | Real cases with checked answers, scored after every change |
| A cost ceiling | "The bill tripled and nobody can say why" | A cap per run, caching for stable context, an easier model for easy cases |
| An action allowlist | "Security cannot approve a general-purpose key" | A named list of operations, parameters and identities |
| A human lane | "It answers the hard cases with confidence, and wrongly" | A review route with a threshold the business can change |
| Integration scope | "The ERP has no usable way to write back" | A connection spike before any promise of dates |
The sections below take each in turn.
1. A test set
If no fixed cases with checked answers exist, each prompt or model change is a gamble. Quality slides, and the first person to notice is usually a user. Then nobody can name the change responsible.
Gather the cases from end users while discovery is still running. About a hundred real questions, each with a verified answer, gives you a start. How to build an AI evaluation set covers the method.
2. A cost ceiling
An agent loop retries a step, revises its plan and reads its context again. With no limit, spend tracks how difficult a case is, not what it is worth. Difficult cases are frequently the ones worth least.
Fixes are ordinary engineering. Bound the loop. Cap every run. Cache context that seldom changes. Send simple cases to a cheaper model. A cap per run replaces a bill with no upper limit by a unit cost that finance can read. Set it before the first invoice arrives.
3. An action allowlist
A broad API key plus a prompt asking the agent to take care does not count as security, and a reviewer will say so. Write the boundary down: which operations, with which parameters, under which identity.
That list is the security model and the thing a reviewer can inspect. Bolting it on after a pilot works means redesigning how the agent calls tools. AI governance for regulated Indian companies shows where it sits beside the other controls.
4. A human lane
Most queues hold many routine items and a few that need judgement. An agent that clears the routine ones and prepares the file for the rest is simple to defend to a risk committee. An agent with no route to a person either gets the hard items wrong or jams the queue.
Make the threshold a business setting that a named person can change. A value hard-coded in the software outlives the person who chose it. Human-in-the-loop design explains how to build the lane.
5. Integration scope
Plan on the links to the ERP, the case-management tool and the document store being the bigger share of effort. A demo hides this because it reads from a sample file. Production needs authenticated reads, safe writes, retries and a way to undo a step.
Run a short spike against the real systems before promising any date. Find out whether a write interface exists, who owns it and whether that owner will change it. Skipping this step can cost quarters, if the build then finds that a core system is closed to writes and its vendor will not open it.
The first use case that survives
A cancelled project is easy to spot early. It looks good in a demo, touches something leadership can see and has no baseline. A surviving project is plainer. Apply four tests:
| Test | Passes when |
|---|---|
| Backlog | The queue has a measurable volume and wait time today |
| Owner | A named manager wants the queue handled |
| Baseline | Today's accuracy, speed and cost per item can be stated as numbers |
| Rules | Correct handling can be checked against a policy or a past decision |
Loan document checks, matching invoices to purchase orders, claims intake and ticket triage tend to pass. AI in loan origination walks through one of them. Without a baseline, you cannot show the agent worked, and a result you cannot show does not win a second round of funding.
Reading the numbers at a review
A review meeting wants one figure. Give it three, side by side:
- Items finished without a person, as a share of the queue.
- Accuracy of those items on the test set.
- Cost per item against the baseline from before the agent.
If the first rises while the second falls, the agent is handling more items worse. If the third rises with no gain in the first two, the loop needs a tighter limit.
Measures to track
Track items finished without a person, accuracy on the test set, hand-off quality and cost per completed item. The enterprise guide to AI agents defines each measure and the straight-through rule in full.
The order that works
- Pick a queue with a baseline and write the number down.
- Gather the test set from the people who work that queue.
- Settle the data boundary and the list of permitted actions.
- Release with a review lane and a log that can replay each run. See the fields an AI audit trail must hold.
- Run beside the current process, not in place of it.
- Measure the result against the number you wrote in step one.
The model underneath will be swapped more than once. The test set, the limits, the allowlist and the log are what you keep. For the wider picture, see the enterprise guide to AI agents.
Signs a project is heading for cancellation
Look for these in a status report:
- Progress is described by demos and not by scores.
- Nobody can state the current cost per item.
- The security review has been postponed twice.
- The pilot users are the project team.
- Integration is listed as "phase two".
Two or more of these in one report is a reason to stop and add the missing control before spending more.
Questions teams ask
Is the model ever the reason?
Sometimes, and the test set will show it. If accuracy stays low on well-formed cases after retrieval and prompts are fixed, a different or larger model is a measured next step.
How do we keep token costs predictable?
Set a cap per run, log the cost of each item and review the spread monthly. Check the longest runs first, since a handful of them can dominate the total.
What share of the effort is the agent?
Treat that as an open question until the integration spike is done. Estimate the connections and the agent separately, and let the spike decide the split.
Should we build the agent or buy one?
Decide per use case. Build or buy for AI sets out the questions.
Where Brihat fits
Brihat Infotech's AI agents and automation service includes cost and safety ceilings, action allowlists and release gating on an evaluation set. Start a conversation.