Flat Priors

In the fashion where a mathematical proof introduces a base case as the first example, we will do this for priors as well.

A flat prior essentially demands that p(theta) is held at a constant value (usually a natural number).

You can use any constant, but this prior that equals one is called the “unity prior”.

So what does having a constant value prior mean to us? It assumes that all parameter values are equally likely to appear in a set of samples from a population.

How does this affect the posterior?

It causes the posterior to be essentially entirely affected by the likelihood alone. The posterior becomes some fraction of the likelihood. If you want to think of the posterior as a normalization of the likelihood, you can, but it is enough to think of it as just a fraction!

When we have a flat prior, the posterior equals likelihood*prior*weight. For a unity prior it is just likelihood*weight

Some argue that the “flat prior” mimics the most basic case — the case in which we can reflect no bias or judgement about the data being used to construct the prior. In other words, this is the most “objective” prior and is championed for corresponding scenarios and scientists. This is a favored type of prior when the goal is for the data to shine.

It can be argued that a flat prior in “uninformative”; however, you can also say the absence of information about a dataset is information in itself ;).

All priors contain information.

So lets put this prior to action!

If we let p(theta) be any constant, then if we take the integral from negative infinity to positive infinity, we get infinity not 1. So, the prior is improper. The prior only makes sense for unbounded values of the parameter in question.

However, we can still construct a posterior. It would just be considered “limiting cases” at best.

Can a flat prior be a problem?

Sure. Flat priors essentially allows for more unusual parameter observations to manifest in the likelihood and then posterior. Flat priors help keep these outliers in “check”, using knowledge of patterns observed in the past.

We must remember, in the likelihood, we are putting some faith in the parameter values that were collected. So if these are biased, a flat prior cannot make up for it.

For situations where we have little data but want a reasonable posterior, we might want a more informational prior, rather than a flat one.

Main Takeaway: Flat priors shift the influence on the posterior to likelihood, and can often conflate values of parameters by not using any previous understanding of the data. This approach assumes that all parameter values are equally likely to appear in a set of observations.

Research done from: “Priors” A Student’s Guide to Bayesian Statistics, by Ben Lambert, SAGE, 2018.