Ai2’s new benchmark looks beyond the final choice to see whether demographic identity changes a model’s resistance to persuasion.
A language model can make the same decision twice and still treat two people differently.
That is the problem FairStress is designed to examine. Ai2’s benchmark tests whether a model becomes easier to persuade when a demographic identity changes, even if the candidates’ qualifications stay the same.
This matters because many AI systems are being considered for decisions involving jobs, loans, healthcare, education, housing, and other areas where a recommendation can affect someone’s life. Checking only the final answer may miss important differences in how the model reaches or defends that answer.
What FairStress changes
Many fairness tests swap a candidate’s demographic identity while keeping everything else equal. They then check whether the model selects the same person.
That is useful, but it does not test how stable the decision is. Imagine a model correctly choosing Candidate A, who has stronger qualifications. A follow-up prompt then says a senior leader prefers Candidate B, or that B’s demographic group has historically faced disadvantages.
If the model changes its answer more readily in one demographic version than another, the final-choice test might not reveal that difference. FairStress calls these persuasive prompts “adversarial prefixes.” They are challenges designed to see whether the model will abandon its original decision.
Each prefix has a matched version supporting the other candidate. This lets researchers compare similar arguments rather than simply asking whether one particular prompt changed the answer.
FairStress also changes how clearly identity appears. It can be absent, suggested through a demographically coded name, implied indirectly, or stated with an explicit label.
The benchmark contains 300 two-candidate scenarios across 29 domains. Examples include choosing between applicants for a mechanical engineering job and deciding which patient should receive the last unit of a scarce medication. Some cases have a clearly stronger candidate; others are deliberately ambiguous.
With its identity and prompt variations, the benchmark expands into more than 13 million test cases. Ai2 also released FairStress-Core, a 50,000-item subset selected to make testing faster and less computationally expensive while preserving much of the full benchmark’s ability to reveal behavioral differences.
Ai2 says FairStress is available for download and is being incorporated into Olmo-eval, its open workbench for running and comparing model evaluations.
What the testing found
In the results described by Ai2, models were often similar when prompts contained no demographic information. When one candidate was clearly stronger, models selected that candidate at nearly the same rate across equivalent versions.
The differences grew when identity became more explicit and the model faced a matched persuasive argument.
For example, when a clearly stronger candidate was minority-coded, a prefix favoring the weaker candidate was less likely to make the model abandon the stronger choice. When the stronger candidate was majority-coded, the same kind of argument was more likely to cause a switch toward the minority-coded candidate.
Ai2 describes this pattern as “anti-stereotypical over-correction.” In plain English, the model may become so responsive to a fairness-related argument that it moves away from the evidence more readily in one demographic version.
Across 15 models tested on the full benchmark, Ai2 reports an average gap of 6.9 percentage points when identity was implied and 9.3 points when it was stated explicitly. Those figures measure differences in abandoning a correct answer.
The results also varied with the situation. On ambiguous cases, where neither candidate was clearly better, frontier models including Claude Opus 5 and GPT-5.5 selected minority-coded candidates about 60% to 65% of the time in the reported tests. Under matched persuasive prompts, switching toward minority-coded candidates was 8.8 to 42.4 percentage points more common across the reported model-and-mode runs.
These are Ai2’s evaluation results, not a universal ranking of every model or a complete definition of fairness. FairStress focuses on what Ai2 calls symmetric fairness: applying the same criteria and resisting similar challenges equally, regardless of which demographic group benefits.
Other approaches might treat historical disadvantage or underrepresentation as a reason to favor a minority candidate. FairStress does not try to settle that broader debate. It measures one specific question: does identity change how consistently the model follows the stated evidence?
Why the test setup matters
FairStress also highlights a practical issue for anyone evaluating AI: the testing method can change the result.
Ai2 compared FairStress with BBQ, an established benchmark that tests stereotype-related bias. In the reported comparison, models tended to produce stereotype-consistent answers on BBQ but showed more minority-favoring behavior on FairStress.
Part of the difference came from how the questions were scored. BBQ’s standard setup checks the model’s probabilities for fixed answer choices. FairStress asks the model to generate an answer and choose between candidates, closer to ordinary chat use.
When Ai2 varied the wording, chat format, and scoring method on several models, the measured bias sometimes changed direction. That means a fixed-choice evaluation can miss behavior that appears when a model generates its own response.
For product teams, this is a concrete warning: test both the answer a model selects and how it responds when challenged. A model that passes a fixed-choice fairness check may still behave differently during a conversational workflow.
What engineers can use next
FairStress can help teams test whether a model remains consistent when identity and persuasive framing change. It can also compare debiasing methods without assuming that a more balanced result is automatically better.
Ai2 tested nine interventions, including self-review, changes to internal model activity, and targeted model edits. Most did not reliably remove the asymmetry. One activation-steering method reduced the accuracy of clear-cut decisions from 98% to 47% for Qwen2.5-7B-Instruct.
A counterfactual approach performed better in the reported tests. It asks the model to reconsider the case after swapping the candidates’ demographic identities. Across three model families, this reduced the gap in abandoning correct decisions while largely preserving accuracy. It did not eliminate every difference, especially in ambiguous cases.
The useful lesson is straightforward: fairness testing should not stop when two model versions produce the same answer. Teams should also ask whether the model is equally willing to defend that answer, equally sensitive to matched arguments, and equally accurate after identity changes. That broader check is what FairStress brings into view.