Interpreting LLMs as Credit Risk Classifiers: Do Their Feature Explanations Align with Classical ML?

A recent study investigates the potential of large language models (LLMs) as alternatives to traditional machine learning classifiers in financial risk assessment. While LLMs have shown promise in various classification tasks, their effectiveness with structured tabular data, particularly in high-stakes environments, is still being evaluated. This research compares LLM-based classifiers with LightGBM, a leading machine learning model, to determine if their feature explanations align. Understanding this alignment is crucial for the financial sector, as it could influence the adoption of LLMs in credit risk evaluation.