The GRIM test

GRIM (granularity-related inconsistency of means) asks one question: could any set of whole-number answers from this many people produce the mean that was printed?

The idea

If each of n people answers with a whole number, their answers add up to a whole number, so the mean is that whole number divided by n. With 28 people the mean must be one of 5.1786…, 5.2143… and so on, in steps of 1/28. Rounded to two decimals, those steps skip some values. When fewer than 100 people are involved, some two-decimal means simply cannot occur.

Worked example

Brown and Heathers open their paper with a made-up study in which 28 people report a mean of 5.19 on a 1–7 item.

5.19 × 28 = 145.32 so the total was 145 or 146 145 / 28 = 5.1786 rounds to 5.18 146 / 28 = 5.2143 rounds to 5.21 no whole-number total gives 5.19 → inconsistent

The same paper's control group, a mean of 3.87 with 27 people, fails the same way (104/27 rounds to 3.85 and 105/27 to 3.89). Try it in the checker.

Scales made of several items

When each person's score is the average of k integer items, the mean of those scores is a whole number divided by n × k. So GRIM still works: multiply n by the number of items. The paper's example has 21 people and 3 items, 63 scores in all; a reported 3.77 needs a total between 237 and 238, and 237/63 = 3.762 while 238/63 = 3.778, so 3.77 is impossible.

When GRIM cannot help

If n × items is at least 10 to the power of the number of decimals (100 for means with two decimals), every value is reachable and the test says nothing. Five items and 25 people already give 125 scores, so means from such a scale are not testable at two decimals. The checker says so instead of reporting a pass.

Rounding ties

Sometimes the exact mean ends in a 5 right after the last printed digit. With 88 people and a total of 275 the mean is exactly 3.125. Rounding half up (what SPSS and Excel print) gives 3.13; rounding half to even (what R's round() and Python do) gives 3.12. Both are honest reports. The checker runs every check under half up, half to even and half down, uses the one you pick for the verdict, and flags any result that would change with the convention. The original GRIM analyses accepted ties rounded either way.

Percentages

A percentage of people is a mean of 0/1 answers, reported with two extra decimals: 29.1% is 0.291. The paper leaves one as an exercise: 29.1% of 847 people. 246/847 is 29.04% and 247/847 is 29.16%, so 29.1% cannot be right. Type the percent sign and the checker handles it.

Source

Brown, N. J. L., & Heathers, J. A. J. (2017). The GRIM test: A simple technique detects numerous anomalies in the reporting of results in psychology. Social Psychological and Personality Science, 8(4), 363–369 (online 2016). doi:10.1177/1948550616673876. Open-access preprint: doi:10.7287/peerj.preprints.2064v1.

Open the checker · GRIM · GRIMMER · SPRITE