Four lessons for moving AI beyond pilots

Sarah Biller is co-founder of Fintech Sandbox, a Boston-based nonprofit that provides entrepreneurs with access to high-quality data sets.

Last month, I moderated a FinovateFall discussion with banking and fintech leaders about a key question confronting nearly every institution: Where is AI producing measurable returns?

The conversation took place under the Chatham House Rule, allowing participants to speak candidly about what has worked, what has not and why many promising experiments stay stuck in pilot purgatory.

The discussion reinforced the point that AI adoption is no longer only about technical capability. It’s also about preparing the organization to use it and defining which business outcomes it should produce.

Here are four lessons that stood out from the discussion.

1. Begin with the business problem, not the technology

The most promising AI applications often begin with an operational problem that is already well understood. Fraud investigations, transaction reconciliation and other exception-heavy workflows are important use cases because institutions can measure.

It also means resisting the temptation to begin with the most novel application. 

We shouldn’t start with the most flashy object. Instead, institutions should choose a manageable problem for which the return can be readily seen and measured.

AI should not be adopted because an executive encountered an impressive product. The business line must first identify the problem, the desired outcome and the other parts of the institution that will be affected.

2. Data readiness can distinguish pilots from production

Financial institutions have spent years collecting enormous quantities of data, but that doesn’t necessarily equal readiness. Data could be fragmented across core systems, business units and external platforms, making it difficult to build a complete view of the customer or deploy AI consistently.

The main challenge comes when the data isn’t AI-ready. Institutions need to understand the quality, location and ownership of their data before attempting to scale an AI application. They also need audit trails that allow them to examine how a model reached a decision.

This work is less visible than launching a chatbot or unveiling a new customer experience. Yet it may determine whether an experiment becomes an operating capability.

3. Measure business outcomes, not AI activity

Employee adoption and use statistics can show whether an AI tool has gained traction, but they don’t always establish a clear ROI.

The more meaningful measures depend on the problem being solved: fraud losses prevented, investigation time reduced, exceptions resolved or additional transaction volume absorbed without increasing staff. The goal may be to “absorb more of the breaks, more of the issues or more of the volume with the existing team,” rather than simply reduce head count.

Institutions should also look beyond the immediate business line. An AI-enabled lending capability, for example, may generate deposits and deepen customer relationships in addition to producing loans.

Costs require equal attention. The most powerful model is not necessarily the right model for every task. Choosing a more computationally intensive model can make the difference between a profitable application and one that consumes more value than it creates.

4. Employees must help build the AI-enabled institution

AI adoption can’t be an isolated technology initiative. You need to engage experts in lending, compliance, fraud and customer service to figure out whether an AI-assisted workflow is useful and trustworthy.

What’s more, it’s on the leaders to explain the institution’s strategy and provide the approved environments for experimentation. They also need to give employees time to participate. One financial institution on the panel said it treats AI development like training, letting employees spend a day a week with its innovation team before returning to their regular roles.

Employees may be more willing to use AI if they understand how it can make their jobs easier without replacing them, particularly by handling time-consuming tasks that require little judgment or experience.

Financial institutions already know how to govern sensitive data. They can evaluate technology partners and redesign business processes. AI requires institutions to adapt those disciplines, not abandon them. The firms that move beyond pilot purgatory will be those that connect the technology to a defined problem and an outcome that matters.