Prior and Likelihood Influence (a visualization)

Let’s use a Bayes Box to illustrate how priors and likelihoods affect the shape of the corresponding posterior distributions!

A Bayes Box is a table that walks you through the calculation of the posterior. There is a column for each part of the equation:

Let’s look back at our apple bobbing example from.

Let’s suppose we have a bucket with 4 apples. These apples can be either red or green.

Let’s use a Bernoulli likelihood where X = 0 means a red apple was caught, and X= 1 means a green apple was caught.

Let’s let our parameter theta be the number of green apples caught.

Then,

for Pr(X=1|0 green apples in bucket) = 0

Pr(X=1|1 green apple in bucket) = 1/5

Pr(X=1|x green apples in bucket) = x/5

illustrates how we construct our likelihood values.

If we want to imitate a flat prior, then out of 4 apples, there is a 1/5 chance that there are either 0 ,1, 2, 3, or 4 green apples in the bucket. Let’s say this is our first time bobbing for apples and we have to previous knowledge to go off of.

Our final Bayes Box then looks like:

Wait, how did we get the last column? What is the denominator?

Remember how we said that the posterior is a normalization of the likelihood*prior?

That is what we did!

To normalize, we sum up the “Prior*Likelihood” column and divide every individual row in that column by the sum. Check to see for yourself.

We notice that the flat prior does indeed produce a proper posterior distribution.

Now for the interesting part, let’s compare graphs of the Prior, Likelihood, and Posterior distributions:

As you can see, with a flat prior, the posterior distribution is almost exclusively impacted by some weighted values of the likelihood distribution.

Let’s be a little more deliberate with our prior. Let’s now assume that there is green apple shortage and having fewer green apples is more likely than all 4 being green.

Our Bayes Box may look a little more like this:

NOW let’s take a look at how the Prior, Likelihood, and Posterior distributions pan out:

Cool! Look at that. The prior seems to have more influence on the posterior distribution now! It sort of balances the extremes of the likelihood distribution, so the posterior shape looks more flat.

Main Takeaway: Bayes Boxes are a great way to organize discretized values of the prior, likelihood, and posterior distributions. Priors that aren’t flat will elicit a greater impact on the final posterior distribution.

**Graphs made in Excel, and Bayes Boxes made in LaTeX.