Bruno de Finetti and Exchangeability
You may recall that likelihoods are constructed from the product of many individual likelihoods.
In order for us to claim this, we must ensure our sample we are basing our likelihoods from is independent and identically distributed or random.
However, this can be a tricky condition to meet.
Thanks to Italian probabilistic statistician Bruno de Finetti, Bayesian may use a similar condition, exchangeability (for large enough samples).
What does it mean for a sequence of random variables to be exchangeable?
Exchangeability: If a sequence of random variables is equally likely as ANY other random variable reordering, then the sequence of random variables is EXCHANGEABLE.
You may be wondering, if a sequence is exchangeable, is the sequence of random variables equally likely?
This is where things slightly fall apart.
Exchangeable doesn’t necessarily take into account independence. For example, you may have a sequence of random variables that whose positions can be exchanged in the context of the entire sequence, but independently, they cannot be exchanged for one another.
Let’s look at an example:
Suppose you were bobbing for apples and there were both green and red apples to choose from. You wouldn’t know which one you picked, because you were blindfolded, but let’s assume the event manager ran out of apples and there were only 4 red and 3 green apples to choose from at the end.
Assuming you get an apple each time, the chances of getting 2 green and 1 red will be the same in any order you get that combination.
This is also true for 1 green, 2 red or any other combination.
As long as the sequence of events is bound by three turns, and there is no replacement of apples, there will be exchangeability.
Exchangeability doesn’t necessarily seem so convenient after all, huh?
The nice thing is that ALL random samples are exchangeable. So, if we can guarantee exchangeability, we can still express likelihoods as a product of other likelihoods in context.
Main Takeaway: Random implies exchangeable, exchangeable doesn’t imply random, and exchangeable is all we need to assume a likelihood is a product of individual likelihoods.
Research done from: “Likelihoods” A Student’s Guide to Bayesian Statistics, by Ben Lambert, SAGE, 2018.
