Finance's Competitive Edge: Trust and Explainability Over "Good Enough" AI

The Unseen Costs of "Good Enough" AI: Why Trust and Explainability Are Finance's New Competitive Edge

In the relentless pursuit of AI adoption, particularly within finance, a dangerous complacency has taken root: the idea that "good enough" AI output is, in fact, sufficient. This conversation with Jeremiah Edwards, Head of Sage AI, reveals the profound, often hidden consequences of this mindset. While the allure of rapid deployment and increased output is strong, the true competitive advantage lies not in sheer volume, but in verifiable trust and explainability. This piece is for finance leaders, CFOs, and SMB owners who understand that a single incorrect number can derail an entire operation. It offers a framework for discerning truly valuable AI from the noise, providing a strategic edge by focusing on accuracy and audibility, not just speed.

The Illusion of Progress: When Speed Outpaces Accuracy

The current AI landscape is characterized by an unprecedented race to deploy more agents, generate more outputs, and achieve more automation. However, this rapid acceleration often comes at the expense of a critical foundation: trust. Jeremiah Edwards highlights that the focus has, for too long, been solely on the model itself, leading to a dangerous oversight of the entire system required for reliable AI, especially in finance. The "dog playing piano" analogy underscores this point: impressive capabilities are meaningless if the core function isn't sound. For finance professionals, this translates to questioning not just if AI can perform a task, but how it arrived at the result and if that result is demonstrably better than existing processes.

The integration of AI into finance workflows necessitates a robust system beyond just a powerful Large Language Model (LLM). Edwards emphasizes that the model must be coupled with an "agentic harness," reliable tools, and, crucially, access to accurate, relevant data sources. The best AI model is ineffective if it's not connected to the definitive source of truth for financial data -- the accounting software itself. This connection, often facilitated by standard protocols, is the bedrock upon which trustworthy financial AI is built. Without it, even sophisticated models are prone to generating incorrect outputs, a risk that is amplified when dealing with sensitive financial figures.

"The best ai model in the world cannot do your accounting for you if it's not connected to the best accounting software in the world -- connected via you know standard protocols like mcp or a to a."

-- Jeremiah Edwards

This system-level thinking is vital because LLMs, despite their advancements, are fundamentally text translators trained on vast, imperfect datasets. The internet, their primary training ground, contains inaccuracies that are difficult to filter. Consequently, these models can struggle with basic arithmetic and factual recall. To overcome this, Edwards points to the necessity of providing AI with access to tools, such as a calculator, to perform arithmetic reliably. This pragmatic approach grounds AI capabilities in verifiable accuracy, moving beyond theoretical model performance to practical, dependable outcomes.

The CFO's Hot Seat: Accountability Demands Explainability

The PwC study, revealing that 71% of finance leaders would reject AI that cannot explain itself, perfectly encapsulates the core challenge. Edwards stresses that the ultimate accountability for financial integrity rests with the human CFO or business owner, not the AI. When audits occur or taxes are filed, it is the human who faces scrutiny, not the algorithm. This inherent accountability elevates the standard for AI in finance; "close enough" is simply not good enough.

This reality mandates that explainability be treated as a "first-class design principle." For Sage, this means every step an AI agent takes -- from data sources consulted to API calls made and the reasoning employed -- must be fully auditable and transparent to the user. This transparency, often manifested as "chain of thought" reasoning, allows users to understand how an AI arrived at an answer. It’s not just about getting an answer, but about seeing the logical progression that led to it, enabling critical judgment and fostering confidence.

"At the end of the day the cfo or a or a small business owner is accountable for for the integrity of their books... it's not claude that's sitting in the hot seat with the irs it's the cfo."

-- Jeremiah Edwards

The concept of "chain of thought" reasoning is crucial here. It signifies a move beyond single-prompt AI interactions to a more iterative process where the AI critiques its own answers, explores available tools, and justifies its actions. This capability, when surfaced to users, is what makes AI usable for CFOs. It transforms AI from a black box into a transparent assistant, empowering finance professionals to review, validate, and ultimately trust the outputs. This focus on reasoning and transparency is not merely a feature; it's a prerequisite for AI adoption in mission-critical financial functions.

Navigating the Transition: Agency, Skill, and the Future of Finance

The rise of agentic AI, capable of performing complex tasks, naturally raises questions about human agency and expertise. For seasoned finance professionals, seeing AI perform tasks they've built careers around can be unsettling. Edwards acknowledges this as a universal challenge, even for technologists. However, he frames AI not as a replacement for human expertise, but as a tool to address talent shortages and elevate the role of finance professionals.

The advice offered is to approach AI adoption incrementally, starting with task-based automation like accounts payable processing. These applications, while perhaps not perceived as "cutting-edge AI" anymore, offer significant time savings and are a comfortable entry point. More importantly, AI can free up finance teams from looking backward at historical data to looking forward, enabling a shift towards "continuous accounting." This allows for more strategic, forward-looking decision-making, a critical advantage in dynamic business environments.

The key to leveraging this shift lies in governance and control. Edwards advocates for a "human in control" paradigm, where AI acts as the doer and the human as the supervisor and reviewer. This ensures that AI systems provide confidence in data and results, while also offering a control plane for critical judgment. The finance function remains accountable, but with more time to focus on high-value strategic tasks. This iterative process of auditing and improving people, processes, and technology is essential in the rapidly evolving AI landscape.

"The finance function is still accountable for the the financial health of the business... that means tasks are going to be a lot more review based ai is is the doer the human is the supervisor the human is the reviewer and you have more time to do that because you're you're not stuck on on kind of you know lower level tasks."

-- Jeremiah Edwards

For SMBs, the perceived risk of adopting AI can be a significant barrier. Edwards counters this by emphasizing the principle of using the "right tool for the right job." While direct use of models like ChatGPT might suffice for ideation or drafting marketing copy, finance demands a higher standard of accuracy and explainability. He suggests starting with specific use cases, such as outlier detection on the general ledger, which acts as a watchdog without altering core financial data. This approach builds confidence and demonstrates value, paving the way for more advanced agentic AI applications grounded in real data and controlled by human oversight. The future-proof strategy involves staying aware of AI advancements, identifying genuine value propositions, and adopting solutions that offer tangible benefits without compromising trust or control.

Key Action Items

• Immediate Actions (0-3 Months):

  • Implement Task-Based AI for AP: Automate invoice processing and data entry to free up immediate capacity.
  • Establish AI Governance Framework: Define clear policies for AI tool usage, data access, and review processes.
  • Pilot Outlier Detection: Deploy AI tools focused on anomaly detection within your general ledger as a foundational trust layer.
  • Educate Your Team: Conduct workshops on AI capabilities, risks, and the importance of explainability in finance.
  • Review Existing AI Tools: Audit any AI currently in use for accuracy, explainability, and adherence to governance policies.

• Medium-Term Investments (3-12 Months):

  • Explore Finance Intelligence Agents: Investigate AI agents designed for specific financial tasks like closing the books faster or budget analysis.
  • Develop Chain-of-Thought Review Processes: Train your team to actively review and understand the reasoning behind AI-generated financial insights.
  • Integrate AI with Core Accounting Software: Prioritize AI solutions that directly integrate with your existing financial systems for data grounding and accuracy.

• Longer-Term Investments (12-18 Months+):

  • Foster Continuous Accounting Practices: Leverage AI to enable real-time data availability and decision-making, shifting from batch reporting to continuous insights.
  • Build Internal AI Expertise: Invest in training or hiring individuals who can manage, audit, and strategically deploy AI within the finance function.
  • Develop a "Human in Control" Culture: Solidify processes where AI supports human decision-making, ensuring human oversight and accountability remain paramount.

• Items Requiring Discomfort for Future Advantage:

  • Challenging "Good Enough" Mentality: Actively push back against accepting AI outputs without rigorous verification, even if it slows down immediate task completion.
  • Investing in Explainability Over Speed: Prioritize AI solutions that offer clear audit trails and reasoning, even if they are not the fastest available.
  • Addressing Shadow IT: Proactively bring AI tools used outside of official channels under governance to ensure control and mitigate risks.