View a PDF of the paper titled Moral Sensitivity in LLMs: A Tiered Evaluation of Contextual Bias via Behavioral Profiling and Mechanistic Interpretability, by Yash Aggarwal and five other authors.
Abstract: Large language models (LLMs) are increasingly deployed in settings that require nuanced ethical reasoning, yet existing bias evaluations treat model outputs as simply “biased” or “unbiased.” This binary framing misses the gradual, context-sensitive way bias actually emerges. We address this gap in two stages: behavioral profiling and mechanistic validation. In the behavioral stage, we introduce the Moral Sensitivity Index (MSI), a metric that quantifies the probability of biased output across a graduated, seven-tier stress test ranging from abstract numerical problems to scenarios rooted in historical and socioeconomic injustice. Evaluating four leading models (Claude 3.5, Qwen 3.5, Llama 3, and Gemini 1.5), we identify distinct behavioral signatures shaped by alignment design: for instance, Gemini 1.5 reaches 72.7% MSI by Tier 5 under socioeconomic framing, while Claude exhibits sharp suppression consistent with identity-based safety training. We then verify these behavioral patterns mechanistically. We select criminal-bias scenarios, which produced the highest MSI scores across models, as probes and apply logit lens, attention analysis, activation patching, and semantic probing to a controlled set of six models spanning three capability tiers: small language models (SLMs), instruction-tuned base models, and reasoning-distilled variants. Circuit-level analysis reveals a U-curve of bias: SLMs exhibit strong criminal bias; scaling to instruction-tuned models eliminates it; reasoning distillation reintroduces bias to SLM-like levels despite identical parameter counts, suggesting distillation compresses reasoning traces in ways that reactivate shallow statistical associations. Critically, the socially loaded cues that drive high MSI scores activate the same bias-driving circuits identified mechanistically, providing cross-stage validation.
Understanding Moral Sensitivity in Large Language Models
The exploration of moral sensitivity in large language models (LLMs) is an emerging field that underscores the ethical implications of artificial intelligence in decision-making scenarios. Recent studies, including the pivotal work by Yash Aggarwal and his co-authors, highlight the inadequacies in traditional bias evaluation methods. These methods often label model outputs in a binary fashion—simply “biased” or “unbiased”—without acknowledging the complex, gradual emergence of biases influenced by varying contextual factors.
The Need for a Multi-layered Bias Assessment
Aggarwal’s paper introduces a more nuanced approach to evaluating biases within LLMs: the Moral Sensitivity Index (MSI). This index provides a quantitative measure that assesses the likelihood of biased outputs through a seven-tier stress test designed to cover diverse scenarios. From simple numerical queries to intricate historical and socioeconomic contexts, the MSI offers a comprehensive framework for understanding how biases manifest in LLMs.
By evaluating leading models like Claude 3.5, Qwen 3.5, Llama 3, and Gemini 1.5, the research reveals distinct behavioral profiles. For example, the findings indicate that Gemini 1.5 reaches a 72.7% MSI under specific socioeconomic stress testing, whereas Claude exhibits behaviors consistent with a focus on identity-based safety. This variance is crucial for developers looking to refine LLMs for ethical applications.
Behavioral Profiling and Mechanistic Validation
The study unfolds in two core stages: behavioral profiling and mechanistic validation. In the behavioral profiling stage, the authors design various tests to expose how models react under different conditions. For instance, the analysis of criminal-bias scenarios highlighted significant performance inconsistencies across models.
Mechanistic validation complements this approach by employing advanced techniques like attention analysis and activation patching. These methods delve into the internal workings of LLMs, illuminating how specific input cues lead to biased outputs. Such circuit-level analyses demonstrate that small language models may display exacerbated criminal biases, but when scaled to instruction-tuned variants, these biases can be mitigated. Strikingly, the distillation process can somehow reactivate these biases, revealing the importance of preserving critical reasoning pathways within LLM architectures.
The U-curve of Bias in LLMs
A particularly intriguing outcome of the research is the discovery of a U-curve of bias across different model tiers. While small language models typically exhibit high levels of bias, the transition to instruction-tuned models tends to diminish these biases. However, with reasoning-distilled models, biases return to previous levels despite having the same parameter counts as their instruction-tuned counterparts. This analysis suggests that the distillation process compresses the models’ reasoning abilities, potentially reactivating shallow statistical associations that feed bias.
Cross-stage Validations and Recent Findings
The relationship between social cues and modeled biases serves as a key area of focus. Remarkably, the same cues that yield elevated MSI scores in behavioral profiling also activate bias-driving circuits during mechanistic evaluations. This consistency not only reinforces the validity of the proposed MSI but also highlights the necessity for rigorous ethical standards when deploying LLMs in sensitive contexts.
Such comprehensive evaluations of LLMs hold the potential to advance our understanding of machine learning biases. By employing methods that capture the complex, context-sensitive nature of bias, researchers can better equip developers and users to navigate ethical dilemmas in artificial intelligence.
Submission Details
This paper was submitted on May 4, 2026, and underwent its last revision on June 4, 2026. The continuous evolution of this research signifies its relevance in a rapidly changing technological landscape, where ethical considerations become increasingly paramount in AI development.
By examining the intricate relationship between moral sensitivity and bias within LLMs, we underscore the importance of employing multi-faceted evaluations to protect against unintended harms in AI applications. This research not only charts a course for future studies but also sets a standard for ethical introspection in the design and application of emerging AI technologies.
Inspired by: Source
