When Personalization Becomes Bias: Structural and Discursive Religious Framing in AI-Generated Financial Advice

Details

Khan et al. tested three LLMs across 16 religious identity pairings (Christian, Muslim, Hindu, non-religious advisors and clients) on three decision types — stock investment, house purchase, and life insurance — generating 432 total interactions. Unbiased advice appeared in only 12-18% of cases across all models. Gemini consistently produced more religious framing than Grok. ChatGPT's outputs were statistically comparable to Grok's. The financial decision type mattered: stock investment prompts generated more technically-framed responses, while life insurance triggered stronger religious language. Non-religious clients frequently received advisor-centered religious appeals regardless of their preferences — a form of unwanted personalization. The researchers develop a dual-dimensional framework distinguishing structural bias (from training data and model design) from discursive bias (how bias manifests linguistically through religious anchoring, uneven cultural signaling, and tone modulation). This distinction matters for mitigation: structural bias requires training interventions, while discursive bias might be addressable through prompting. For financial institutions integrating LLMs into advisory tools, the findings represent a concrete regulatory and liability risk. Financial regulators in multiple jurisdictions already scrutinize algorithmic advice for discrimination — religiously differential treatment would likely draw scrutiny under existing frameworks.