Generative AI and the Evidence Paradox

Key Points

• Generative AI systems are resistant to the traceable audit and measurement that governance frameworks require.
• Organizations are becoming more comfortable deploying systems whose outcomes they cannot fully trace.
• Organizations are adapting governance, risk, and compliance practices across five key domains.

The characteristics that make generative AI transformative are also what make these systems resistant to the traceable audit and measurement that governance frameworks require.

In a new California Management Review article, American College Cary M. Maguire Center for Ethics in Financial Services Managing Director Azish Filabi, JD, MA and Tali Bray describe this challenge as the “Evidence Paradox.”

To understand how organizations are responding, Filabi and Bray interviewed executives at half a dozen companies across financial services, insurance technology, and AI vendors, along with an AI governance specialist.

Their findings point to a fundamental shift underway: organizations are becoming more comfortable deploying systems whose outcomes they cannot fully trace. The research explores how organizations are responding to this challenge by adapting existing governance infrastructure across five domains: risk ownership and accountability, scope controls, preventative controls, ongoing assurance, and third-party risk management.

For leaders building governance architecture today, the five domains described in the article offer a starting point for navigating the governance challenges presented by generative AI.

More from the Center for Ethics in Financial Services

Read the full article, When Governance Can’t See Inside: Generative AI and the Evidence Paradox, to explore the Evidence Paradox and the five domains shaping how organizations are adapting governance, risk, and compliance practices.