Fairness testing for AI decisions: start with the four-fifths rule, do not stop there

In short: the four-fifths rule compares each group’s selection rate with the highest group’s rate; a ratio under 80 percent is a signal of possible adverse impact. It is a quick screen, not a legal safe harbor: small samples mislead, a statistically significant gap can matter even above 80 percent, and newer laws expect audits, records and a look at proxies. Use it first, then test properly.^1,2

When an AI system helps decide who is hired, promoted, approved or served, the first question a regulator, a court or an auditor will ask is whether the outcomes differ across groups. In US employment practice the first screen for that has been the same since 1978: the selection-rate comparison in the Uniform Guidelines on Employee Selection Procedures, usually called the four-fifths rule.¹

How the rule works

Compare the selection rate of each group with the rate of the group selected most often. If a group’s rate is less than four-fifths (80 percent) of the highest rate, that is generally regarded as evidence of adverse impact that needs a closer look.¹ If 50 percent of one group passes an AI screening step and 35 percent of another, the ratio is 70 percent, below the threshold.

Group C shows why the ratio alone is not enough: with 40 applicants, two or three different outcomes would move it across the line. Pair the ratio with a significance test and a look at sample size before drawing conclusions.

Why it is a start, not a finish

• It is a screen, not a verdict. Falling below 80 percent does not prove discrimination, and passing does not prove fairness. Small samples in particular produce ratios that move a lot by chance. The EEOC’s 2023 technical assistance on AI selection tools called the rule a “rule of thumb” and said a tool could still have unlawful adverse impact if it produced a statistically significant difference in selection rates.² In January 2025 the EEOC pulled a number of its AI-related publications from its website after a change in federal AI policy; the anti-discrimination law they described did not change.³
• It looks at outcomes, not causes. A model can pass on overall outcomes and still rely on proxies for protected characteristics, as Amazon found with a recruiting model that learned to penalize the word “women’s.”⁴
• Laws now ask for more. New York City requires an independent bias audit before an automated employment decision tool is used for jobs in the city.⁵ California’s Civil Rights Council regulations, in force since October 1, 2025, treat evidence of anti-bias testing, or its absence, as relevant to discrimination claims and require four years of records.⁶

Beyond hiring

The rule comes from employment, but the method travels. Lenders compare approval rates, insurers compare pricing outcomes, and public services compare access rates across groups, each under its own law and with its own statistical expectations. Where no rule names a threshold, the habit is the same: decide the comparison in advance, measure it on real decisions, and write down what you did when a gap appeared.

A testing routine that holds up

1. Choose the measures with legal and the business before testing, not after a result you dislike. Write down why.
2. Test before anyone relies on the output, on data that reflects the people the system will actually assess.
3. Test again on a schedule and after every change to the model, the data or the process around it.
4. Look behind the numbers. When a ratio falls short, examine which features drive the difference and whether they are proxies.
5. Keep the records: the data, the measures, the results and the decision taken. Under several laws they are now evidence.

If your organization uses an AI tool supplied by a vendor, ask the vendor for its testing evidence, and run your own test on your own applicants. The legal responsibility for the decision stays with you. Colorado’s new law, described in our briefing on SB 26-189, dropped the statutory duty to prevent algorithmic discrimination, but federal and state anti-discrimination law still applies, and the Rite Aid order shows what happens when no one tests outcomes across groups.

Bias, fairness and the duty to explain are the first lesson of Module 7 of the AIMP program, with a worksheet for exactly this calculation. The four-fifths rule is nearly fifty years old; used as the first step of a routine rather than the whole of it, it still earns its place.