The staircase guess
If everyone really were reading the same ladder, you’d only need one number per
person to reproduce all their answers: how many times they said yes. Someone with 23 yeses
would say yes to the 23 most popular things and no to everything else. That pattern is
called a Guttman scale, after the sociologist who came up with it, and it’s
the grey staircase in the chart at the top.
Make that guess for everyone and check it against what they actually said: it’s
right — of the time.
Why that’s less impressive than it sounds
Most of the fifty things aren’t close calls. Nearly everyone said yes to the dog and
no to the couch, so you can get a lot right without knowing anything about anybody: just
guess the majority answer for every item. That gets —.
So the fair question is how much of the remaining room the staircase captures.
Of the gap between — and a perfect 100%, it closes
—. (This is the coefficient of
scalability, —.) A little over half the room you
could improve on, from one number per person.
Counting skipped steps
Another way to check: look for skipped steps. When someone says yes to something lower
on the ladder but no to something higher — R4, for instance, said yes to a fictional
character (six votes) but no to a clam (twenty) — they’ve skipped a step.
Some of that happens by chance, and given how popular each item is, you can work out how
many skipped steps to expect if everyone were answering at random.
This survey has — fewer skipped steps than
random answers would produce. That’s Loevinger’s H
= —, and anything above 0.5 is conventionally called a
strong scale.
Do the same count one item at a time and each item gets its own score. Items under 0.35
are marked off-ladder: they skip steps nearly as often as random
answers would (the sleeping and anesthetised person, the ant colony, the United States), or
more often (Sherlock).
Could one simple rule do better?
I also wrote down seven simple rules — is it alive? does it have a brain? —
and scored each one against every answer, as if everyone used that rule. None of them even
beats guessing the majority.
Look at the kinds of misses. “Nervous system” and “animal” almost
never say yes when people said no; they just miss an enormous number of yeses. People grant
experience to far more than those rules allow, not less.
Are there camps?
To check whether people fall into schools of thought, I looked at each person’s
surprises: the answers where they broke from their own staircase guess. Mark a
surprise yes as +1, a surprise no as −1 and everything else as 0, and each person gets
a fifty-item “quirk profile”.
If there were camps — say, a group who all count electricity — their quirk
profiles would line up: surprising on the same items, in the same direction. I measured that
with a correlation between every pair of profiles, where 1 means identical quirks, 0 means
unrelated and −1 means opposite. Averaged over all —
pairs of people, it comes out at —.
To see what “unrelated” looks like with this little data, I shuffled each
person’s quirks across the fifty items, — times
over. That keeps how quirky each person is but destroys anything they share. The shuffled
average is —, so the real quirks line up, if
anything, slightly less than chance.
And the most alike pair of people? —, with a
correlation of —. The most alike pair in shuffled
data averages —, and beats
— — of
the time. —
Small print
The shading on the dots is just the generosity order cut into thirds
— a reading aid, not groups.
One typo. The form asked about “a single blade of
glass”, which was meant to be grass, and it’s read as grass
here. It got —, right where grass should land, so
people almost certainly read it the same way.
— people is not many. The staircase is far too strong to be
chance, but any single 7-versus-8 split is a coin toss. Trust the shape of the ladder, not
the individual rungs.