The five-people test: how to tell whether AI in finance actually worked
Most firms can tell you their AI is working. Far fewer can tell you what changed. Alain Mandy, who spent 25 years in institutional asset management before advising DwellFi, has a harder test.
By Shivani Kaul and Martina M.
For most of Alain Mandy's career, the monthly market commentary took three days. An analyst pulled the fund's performance and the benchmark, worked out what had happened and why, wrote it up, and did it again the next month, and the month after that.
Feed the same two inputs to a model now and the commentary comes back in ten minutes. Mandy describes this without much drama, because by the standards of 2026 there is not much drama in it. This is the part of AI in finance that is no longer in dispute.
The question he is more interested in is what happened to the three days. If they were reabsorbed into the same analyst's calendar, the firm bought time, and time has a habit of filling itself. If the commentary now covers three times as many funds without a third analyst, the firm bought capacity, and capacity shows up in the accounts. Same technology, same output, two completely different outcomes, decided entirely by what the firm did next.
That decision is rarely made deliberately. It is usually made by default, which is how a firm arrives at the end of a year with high adoption, an unchanged operating model and a larger cost base.
Mandy spent 25 years inside institutional asset management, latterly as a managing director at Wellington Management and before that as an executive vice president at PIMCO. He worked in operations throughout, which means he spent most of his career on the side of the table that signs off on numbers rather than the side that sells software. He is now a senior advisor to DwellFi. He is not sceptical about the technology. He is sceptical about the accounting, and his test is blunt.
“If today you have five people in the company doing a job and they all use AI, and in one year we still have five people doing the same job and they use AI, it's no gain for the company. What is the benefit for the company? You basically bought an AI tool which is very expensive, and now you have the token, which is very expensive.”
Why AI in finance keeps producing efficiency instead of returns
The failure he is describing is recognizable, and it never looks like failure from the inside. Twelve months into a programme, adoption is high: the fund accounting team drafts LP emails with it, summarizes subscription documents with it, and no longer sends anyone to take meeting notes. People say it saves them time and they mean it. Then someone asks what the close costs now, and it costs what it cost last year, staffed by the same people, on the same timetable, with a licence fee underneath it.
The pattern is visible in the data. McKinsey's 2026 global survey found that 37% of respondents attribute at least some EBIT impact to their organization's use of AI, which is roughly the same share as the year before. Adoption climbed. The line that measures whether any of it reached the accounts did not move.
The same survey found something more useful about the firms where it did. Nearly three-quarters of high performers report fundamentally redesigning workflows because of AI, against about a quarter of everyone else. The dividing line is not model quality or budget. It is whether the work itself was rearranged.
Mandy puts the failure mode in terms an operations lead recognizes immediately.
“If the employee working nine hours per day works six hours per day, that doesn't make a difference for the company. Maybe it makes a bit of difference for the people.”
This is not an argument against assistants. Drafting help is genuinely useful, and a team that spends less of its evening on formatting is a team more likely to still be there next year. The problem is what gets reported. Personal efficiency and operational change look similar in a steering committee deck and behave nothing alike in the ledger. One shortens a person's day. The other changes what the firm can do with the same number of people.
Which fund operations workflow should you automate first?
Mandy has a rule for choosing, and it has nothing to do with which workflow is most painful or most talked about.
“If you do a process once a year, you might not automate it with AI. It might be better that once a year people can do that. It takes maybe half a day, and that's okay. But if you do the same process every day, it makes a lot of sense to automate that process.”
Frequency, in other words, before glamour. It sounds obvious and it is routinely ignored, because pilots tend to get selected by visibility. The annual regulatory filing that a controller dreads is memorable. The daily position and cash reconciliation that three people work through every morning is not memorable at all, and it is where the hours actually are.
Run the rule across a private markets operations calendar and the shortlist writes itself. Daily reconciliation across bank, custodian and general ledger. Capital call and distribution notices, generated against the terms in the LPA and repeated every quarter across dozens of vehicles. Investor onboarding documents. Tax package extraction during K-1 season, where the volume arrives all at once and the logic barely varies from one document to the next. These share the two properties that make automation worth the integration cost: they repeat, and a correct output is defined rather than debated.
The half-day annual task fails the test. Leave it alone.
How accurate does AI in fund operations have to be?
Ask Mandy where the bar sits and he answers from the seat of someone who used to sign off on the numbers.
“In operation, 100% doesn't exist. People make mistakes. But you need to be 99.9%.”
DwellFi states the same standard as a design principle: these are workflows where 100% accuracy is the only acceptable outcome. Mandy is describing what that principle has to survive once it meets a real operation. Manual fund accounting has never been error-free either. What makes it trustworthy is that the operation is built so mistakes surface, through a second pair of eyes, a tie-out, a review, an audit. A system earns its place by holding to the standard and by being built the same way underneath.
So the useful question about accuracy is not how often the system is right in a demo. It is whether you find out when it is wrong, and whether you can prove afterwards how it got there. A confident, plausible, unverifiable number is the dangerous output, because someone will sign it. DwellFi has written at length about why deterministic execution and source-linked outputs are what make that possible, so the mechanics are not worth repeating here. The point for the five-people test is narrower: work only leaves a team when the team can defend the output without redoing it. Anything less and the review step quietly rebuilds the headcount you were trying to release.
What happens to the people
Mandy is direct about this in a way that vendors usually are not.
“Especially in fund administration, in two to five years all the tasks they are doing will be replaced by AI. But again, I'm a strong believer that you still need human in the loop.”
He expects fewer people in fund operations. He does not expect zero, and he is specific about what the remaining roles do.
“You're going to need people to orchestrate all those agents and review what they do and make sure they do the good job, but also manage those agents. And that's going to be the most difficult one.”
Note which way round that is. Supervising a set of agents through a NAV close is harder than performing the close, not easier. It requires knowing what the answer should look like before you see it, which is exactly the judgement that fund accountants spend a decade acquiring. The skill that depreciates is the craft layer, the deep Excel and the inherited macro workbook that only one person fully understands. The skill that appreciates is the review.
This is also the real constraint on how fast any of this happens. Mandy's own read on adoption is that the industry has stalled one level too high: firms have made individuals more productive, and almost nobody has reached the state where agents run through the night and the pack is ready before anyone logs in. The gap is not model capability. It is that few operations teams can yet supervise an agent with the confidence they supervise a junior.
What an AI-native fund operation actually requires
Which brings the argument back to architecture, from an unusual direction. The design choices that get marketed as compliance features are the same ones that decide whether work can leave a team at all.
Data has to be readable where it already lives, because a migration project is how an eighteen-month deployment becomes a thirty-month one. Processing has to happen inside the firm's own environment, because private markets data, meaning LPA terms, side letters and investor records, is frequently not permitted to leave it. Calculations that carry money have to produce the same output from the same input every time. Every figure has to carry a citation back to the document that produced it. Consequential actions have to wait for a named human to approve them.
DwellFi builds to that specification. It describes itself as a sovereign agentic OS for fund operations: single-tenant deployment inside the customer's own cloud account or on-premise infrastructure, data indexed in place, deterministic execution for the arithmetic, line-by-line citations on extracted values, and approval gates before anything consequential executes. It runs against the workflows that pass Mandy's frequency test, including reconciliation, NAV close, capital calls and distributions, waterfall and carry, KYC and investor onboarding, and fund conversion.
None of that is what makes a deployment succeed. It is what makes success measurable, which is a lower claim and a more useful one.
The test worth taking into your next AI conversation
The question to ask about AI in finance is not whether the technology works. It does, more of it every quarter, and the demos are no longer the hard part.
The question is the one Mandy asks. A year from now, when the same workflow comes round again, how many people will be doing it? If the answer is the same number, slightly happier and slightly faster, the firm bought a subscription. If the answer is fewer people, or the same people running four times the volume, the firm bought an operating model, and the economics of fund administration start to look different.
Everything else, including the choice of vendor, follows from being honest about which one you are buying.
If you want to test this against something real rather than a slide, bring one workflow you run every week and a set of sample data, and DwellFi's team will run it end to end with you.
Key takeaways
- Adoption is not a result. The measure of an AI deployment is whether work left the team, not whether the team feels faster. Personal efficiency and operational change look alike in a board pack and behave differently in the ledger.
- Choose the first workflow by frequency, not visibility. A daily reconciliation is worth automating. An annual filing that takes a controller half a day is usually not, however irritating it is.
- Hold the 100% standard, and build for verifiability underneath it. What makes any operation trustworthy, manual or automated, is that errors surface and every figure traces back to a source. Ask a vendor how you would find out it was wrong.
- The surviving role is supervision, and it is harder than the work it replaces. Reviewing and managing agents through a close requires knowing what the answer should look like in advance. That capability, not model quality, is what currently limits how fast firms can move.
Frequently asked questions about AI in finance
What is AI in finance actually being used for today?
In practice, most current use is assistive: drafting, summarising documents, answering questions about internal material, and taking meeting notes. The operational tier is where the economics sit, and it is less widely deployed. That means document extraction, reconciliation, NAV close support, capital call and distribution processing, investor onboarding and regulatory reporting, where outputs are defined, repeatable and checkable.
How do you measure return on AI in fund operations?
By what changed in the operating model, not by satisfaction scores or estimated minutes saved. Useful measures include accounts or funds serviced per person, cycle time for a full close, exception volume against a manual baseline, and rework rate. Set the baseline before the pilot starts, and count the licence and inference cost against it.
Which workflow should a fund administrator automate first?
The one that runs most often and has the clearest definition of a correct answer. For most teams that is daily or monthly reconciliation, or document extraction at volume. Run the pilot in parallel with the manual process for at least one full cycle so the comparison is real.
Does AI in private markets have to be 100% accurate?
One hundred per cent is the right design standard for anything that touches money, and it is worth insisting that a vendor builds to it. It is not an achievable description of any operation, human or automated. The practical requirement is a system that produces the same output from the same input, flags ambiguity instead of guessing, cites its sources, and stops for human approval before consequential actions.
Will AI replace fund administrators?
Alain Mandy's expectation is fewer people doing meaningfully different work within two to five years, rather than an empty function. Task execution moves to agents; oversight, exception handling, client judgement and agent management stay with people, and those roles demand more operational knowledge than the ones they replace.
What does sovereign deployment mean, and why does it matter here?
It means the system runs inside infrastructure the firm controls, with data, model outputs and audit logs staying in that environment. For private markets operators the driver is usually contractual as much as regulatory: limited partnership agreements, side letters and investor records often cannot be processed on a vendor's servers. Sovereign deployment removes that objection at the architecture level rather than through policy.
Put your workflow to the five-people test
Bring us one recurring fund operations workflow and a sample dataset. We’ll run it end to end with DwellFi and show you where deterministic automation can reduce manual work, increase capacity, and preserve human oversight.