When Should I beg to differ?
When I reflect on my life, I realise there are several instances where being “contrarian” actually resulted in very desirable outcomes. These decisions often matter disproportionately during pivotal moments (Whether to drop out, what to study at university, what career paths to go down).
This morning, pondering on this question reminded me of Cellular automata (specifically Conway’s Game of Life) [1] , Complexity and Simulating swarming and herding behaviour amongst social creatures (Reynolds Flocking algorithm) [2].
One thing that this class of algorithms share in common is the notion of emergent complexity.
This is the idea / philosophy that very complex systems can be represented as the manifestation of simple, local rules.
With GOL for instance, we have complex patterns and stable states that are all derived from 4 simple rules which are applied locally (per grid cell).
Reynold’s model of herd behaviour in birds of flight is also super interesting. He was able to simplify this behaviour as 3 rules; Separation, alignment and cohesion. All which are applied locally (per bird).
Humans are not exempt from the herding phenomenon [3]. This isn’t necessarily a flaw.
We are social creatures and herding can be attributed to some of the most important things that allow us to function optimally; FOMO, the desire to belong, etc. Group consensus / convergence also allows for rapid decision making.
However, this very dynamic / phenomenon is the reason financial bubbles / crashes happen for instance [4].
One may ask, how does an independent observer determine a real-world estimated “rate of herding”.
First of all, this has to be measured for a single opinion as “Herding” is not a global measure.
One possible definition (and probably the most intuitive) is Shannon Entropy [5].
Entropy is the measure of instability in a system. Shannon Entropy measures the average uncertainty or surprise in a system. Or in Information theoretic terms; The number of bits required to identify an outcome.
\[H(X) = -\sum_{i=1} p(x_i) \log_2 p(x_i)\]When Entropy is High, the system is close to pure randomness. Total noise. In our case, high entropy means diverse opinions.
In one look, it may seem counterintuitive. If everyone has different opinions, surely that overcomplicates decision making.
Quite the opposite in my view. Let's consider the opposite case:
Everyone collapses on a certain response (low entropy). This implies the conditional probability of a given response given context is the exact same as without context.
\[p(x_i) = p(x_i | \text{context}) \]This implies that X is actually statistically independent of C. Which also implies that the response is the default regardless of context which is an obvious fallacy.
This means we must be aware of the distribution of responses we receive for questions we seek advice on. This is non-obvious.
Other axes necessary for deciding to be contrarian is the risk-taking and experience axes.
This is somewhat similar to the idea of stochasticity and entropy however, more closely related to the exploration-exploitation strategy in Reinforcement Learning that facilitates escape from local optima.
I imagine three graphs: