## Monday, April 06, 2009

### Perceiving randomness: egalitarian bias

After a long time, I am going to agree with a text written over at Cosmic Variance:
Perceiving randomness
Look at the following two pictures. Click to zoom in. We will call them the Ludolf map:

and the Coulomb map:

Which of them is random? Well, both of them look random, according to some distribution, so let me be more specific. Which one is more typical a representative of the probability distribution that has random, uniform, and independent coordinates {x,y} of each point over an interval?

Many people would say that the Coulomb map is more likely to be random than the Ludolf map. The Ludolf map seems way too clustered. In reality, it is the Ludolf map that is random, of course. Repetitions do occur. People usually suffer from the "egalitarian bias" and "psychological quotas".

People love to "balance things" much more than Nature does. People love to say that a sequence of random digits contains every possible value with approximately the same frequency. Nature loves to make the frequencies unbalanced. People love to make random maps uniform. Nature loves to include clusters and voids. Why do the people do this mistake? Well, it's probably because they secretly struggle for "perfection" and they believe that uniformity is similar to "fairness" and therefore "randomness".

Now a secret arrives. If you want to get to a higher level of abstraction and accuracy, none of the maps above was actually random. The Ludolf map was generated from pi. It's enough to say that the coordinates of the points were {314,159}, {265,358}, and 998 others. :-)

I called it the Ludolf map because in Czechia and Germany (and perhaps other countries), "pi" is referred to as Ludolf's number, after the Dutchman Ludolf van Ceulen who calculated 20 digits of "pi" in 1596. If you were a crazy genius (or if you saw my Mathematica notebook in advance), you could have said that the map was derived from "pi".

But if you only look at the statistical distributions and neglect many/most of the microscopic details, "pi" is indistinguishable from a sequence of random digits.

To be sure, the Coulomb map is not random, either. I started with the Ludolf map, gave electric charges to all the points, and moved all of them away from the three nearest neighbors, with some coefficient, in the direction of the Coulomb repulsive force. ;-) I repeated this step thrice or so.

Someone has conjectured that the schools are training the students so intensely that the new generations are likely to develop the opposite, anti-egalitarian bias. I kind of doubt it. The anti-Coulomb map, obtained by the same procedure (one step only) from the Ludolf map, but with the opposite sign, looks like this:

The clustering gets even worse if you repeat the procedure several times. I doubt that people will ever naturally think that such a map is "random" without correlations. Of course that they can be trained to avoid the bias or switch to the opposite bias. But the experience seems to say that people always give up their training and real insights and return to instincts and myths after some time.

The egalitarian bias is here with us to stay which is why the human society is likely to remain less efficient than Nature in many respects.

Bonus: a random image

Click the image to zoom in.

The Fourier modes were chosen randomly, with a normal distribution, suppressed by a power law of the absolute value of the momentum. Yes, the picture is periodic in both directions; you can use it as a background image.

The exponent was chosen 1.5; the value 0.0 would give white noise (random independent pixels). The resulting three matrices (for R,G,B) were remapped to the intervals (0,1) via the Tanh function; the scaling coefficient inside Tanh was chosen to be 1.1.

You may draw your own pictures with a Mathematica notebook. I was surprised that the discrete Fourier transform was actually faster than all the other major steps in the creation of the picture.