Who is accountable when a certified AI agent goes wrong

Eltropy, a digital communication platform used by more than 750 credit unions and community banks in the United States, has opened an Early Access Program that lets fintech builders create and distribute AI agents to those institutions through what it describes as an app store model. Every agent runs inside the company's Safe AI framework under a single compliance baseline.

The comparison invites an obvious question. App stores concentrate power in whoever runs the store, and Eltropy certifies the agents, sets the baseline and sells its own agents on the same marketplace. In written answers to The Datatech Times, Saahil Kamath, head of AI and vice president of product at Eltropy, and Catherine York Powers, founder and chief executive of Constant AI, one of the first builders in the programme, address that question, where liability sits when an agent fails, what certification actually tests, and which of loan servicing, collections and fraud the industry is least ready to hand over.

A neutral operator

Kamath's answer to the concentration question starts with the standards. "Builders want transparency, consistency and choice in a fair marketplace. Eltropy plays a neutral role. We establish the technical, security, operational, and compliance requirements for an agent within a regulated financial institution. The same standards should apply whether the agent is built by Eltropy, a fintech partner, or another developer."

The marketplace, he says, is meant to reduce friction rather than eliminate competition. Credit unions should be able to choose the best agent for a specific need, understand what data it can access, control the actions it is authorised to take and replace it without rebuilding their technology stack, while builders get a shared orchestration, integration, distribution and governance layer instead of recreating connectivity to every core banking, card, lending, payments and authentication system. "Ultimately, Eltropy earns trust only by making the rules clear, applying them consistently, and ensuring that the financial institution, not the marketplace operator, retains control over which agents are activated and how they are used."

Put the sceptic's version to him, that the answer to too many vendors is not a single gatekeeper an institution now depends on entirely, and Kamath argues the risk already exists in a less visible form. "Credit unions already face concentration risk today, but it often appears in a more fragmented and less visible form. Every vendor brings its own integrations, security model, permissions, data practices, monitoring tools, and contractual framework." An institution may work with many vendors and still have no unified view of what each can access or do. "Fragmentation does not automatically create independence; in many cases, it creates unmanaged dependency."

The choice, in his framing, "is not simply between many vendors and one gatekeeper. It is between managing dozens of vendors independently under inconsistent controls, or governing them through a common layer that provides greater visibility and control. Concentration risk is real, and it must be actively designed against by ensuring Eltropy remains an orchestration layer the institution controls, rather than a closed ecosystem it cannot leave."

Where liability sits

On a certified agent causing a bad outcome at a credit union, Kamath declines to put liability in one place. "Liability cannot sit automatically with one party simply because an agent was certified. It depends on what caused the failure, agent configuration, what contractual commitments each party made, and whether the appropriate controls were followed." In his description the builder is accountable for its agent's performance, security and behaviour, including whether it operates as represented; Eltropy for the integrity of the marketplace, the certification process, the orchestration layer, access controls, monitoring and enforcement of the standards; and the credit union for deciding the use cases, the authority granted to an agent, the data it may access and where human review is required. He calls it a shared responsibility.

What the Safe AI framework guarantees, he says, is that "an agent has been evaluated against a defined baseline of technical, security, governance, and operational controls", with visibility into what it can access, what actions it can perform, how it is monitored and how exceptions and low-confidence situations are handled. "What it does not guarantee is that an AI agent will never make an error, that every outcome will be correct, or that certification constitutes regulatory approval. Certification does not remove the institution's responsibility to approve the use and establish appropriate oversight."

Certification itself tests how an agent handles sensitive data, what systems and actions it can touch, how identity and permissions are managed and whether every decision leaves an auditable trail. "Just as important: can we rein it in or shut it off the moment something goes wrong?" An agent would be rejected for failing the compliance or security baseline: careless handling of member data, vagueness about how data is used or retained, permissions that are too broad, no audit trail, weak identity controls or no way to hand off to a human. Certification is also specific to the use case, he says; an agent acceptable for answering general questions may not meet the standard for moving money, changing account information or making lending recommendations. Eltropy does not say which agents or companies have been rejected, because the evaluations involve confidential technical, security and commercial information. "But, the important point is that certification has to be meaningful: if an agent does not meet the baseline, it should not be permitted to operate through the marketplace until the issues are remediated."

The builder's side

York Powers builds inside that framework. Constant AI works in loan servicing and loss mitigation, so its agents deal with members in financial difficulty, and her list of what she will not let an agent decide is long. "At Constant AI, we believe agents should not access anything that requires judgment about a member's financial hardship, the financial institution's rules, disclosures, and writing back the transaction. Most servicing and loss mitigation requests are regulated for a reason. That is not a place for an agent to improvise language or infer a reason. It has to match what is disclosed and documented, every time."

That, she says, is why the Eltropy and Constant AI integration is built the way it is. "Eligibility rules, the exact terms and conditions that have to be read verbatim, and the write-back to the core system are all part of the infrastructure powering the agent, not something the agent reasons about in the moment. There is no agency in that layer. The agent executes within boundaries set upstream by us and the credit union, so the outcome for the member is consistent and compliant regardless of who or what handled the call."

Asked for the honest trade-off of building inside someone else's governed framework, she names two. "Partnering with Eltropy gave us speed to trust and distribution we could not have built alone in the same timeframe. Like any reseller relationship, though, it comes with tradeoffs. As a standalone product, we control how a financial institution understands what we do. Inside someone else's platform, we are one layer in their story." Part of the job, she says, is working with partners early so that powering the agent in a compliant way is not flattened into a feature of the platform, a tension for any startup plugging into a bigger one. The second is pricing. Discounting strategies differ from company to company, "and we want to make sure we are never the loss leader in the overall sale."

Keeping pace, and the losses nobody automates

Both were asked whether AI will really let credit unions and community banks keep pace with large institutions. Kamath's version is the broader one. "A credit union or community bank can now offer round-the-clock service, personalised experiences and sharper financial guidance, without the technology budget of a national bank." AI lets them scale the trust and relationships they already have rather than become smaller copies of a large bank, he says. It may not erase every difference in capital, scale or market reach, "but it can dramatically narrow the technology and operational gap."

York Powers draws the line more tightly. "The comforting story that AI lets community institutions punch above their weight is true on the front end, more limited on the back end." A member calling a 400 million dollar credit union at 9pm can now get the same instant execution of an identity check, an eligibility check or a skip-a-pay as a megabank's customer, she says. "That is bought, not built, and available today." The harder part is elsewhere. "The real gap is trust, and it is closing, just not as fast as the technology." She cites a 2026 CSI survey in which 59 per cent of credit union leaders said they trusted AI-driven decisions against 78 per cent of bank leaders. "Right now, the technology is often ready before the institution is."

On which of loan servicing, collections and fraud the industry is least ready to hand to an agent, the two answer differently. Kamath picks fraud as the area where institutions will move most carefully and where AI may create the most value, with authority expanded gradually rather than granted on day one. "An agent might begin by detecting, investigating, and recommending, then progress to taking narrowly defined actions when the evidence is strong and the controls are clear."

York Powers's answer is about what is not being built at all. "Fraud gets the headlines: deepfakes, voice clones, adaptive attacks, and a whole funded vendor category building explainable AI to fight it. Charge-offs are just as costly, and nobody talks about them the same way. NCUA's own data shows a net charge-off ratio of 81 basis points on $1.73 trillion in credit union loans as of Q1 2026, which implies system-wide losses in the range of $14 billion a year, close to the $16 billion Americans reported losing to fraud in 2025, per the FTC."

"Two comparably sized problems, and only one of them has a mature AI category built around it. The industry cares as much about member hardship as it cares about fraud. The tooling just has not existed until now." Skip-a-pay, deferments, due date changes and payment reductions have stayed manual, she says, while fraud detection has AI agents, explainable models and venture capital. "We built Constant AI for that gap specifically: applying the same rigour to hardship relief that the industry has already applied to fraud, so a $14 billion problem stops being the one nobody automated."

Eltropy announced the Early Access Program on 8 July 2026. Constant AI is among its first builders.