Flat Priors

In the fashion where a mathematical proof introduces a base case as the first example, we will do this for priors as well.

A flat prior essentially demands that p(theta) is held at a constant value (usually a natural number).

You can use any constant, but this prior that equals one is called the “unity prior”.

So what does having a constant value prior mean to us? It assumes that all parameter values are equally likely to appear in a set of samples from a population.

It causes the posterior to be essentially entirely affected by the likelihood alone. The posterior becomes some fraction of the likelihood. If you want to think of the posterior as a normalization of the likelihood, you can, but it is enough to think of it as just a fraction!

When we have a flat prior, the posterior equals likelihood*prior*weight. For a unity prior it is just likelihood*weight

Some argue that the “flat prior” mimics the most basic case — the case in which we can reflect no bias or judgement about the data being used to construct the prior. In other words, this is the most “objective” prior and is championed for corresponding scenarios and scientists. This is a favored type of prior when the goal is for the data to shine.

It can be argued that a flat prior in “uninformative”; however, you can also say the absence of information about a dataset is information in itself ;).

So lets put this prior to action!

If we let p(theta) be any constant, then if we take the integral from negative infinity to positive infinity, we get infinity not 1. So, the prior is improper. The prior only makes sense for unbounded values of the parameter in question.

However, we can still construct a posterior. It would just be considered “limiting cases” at best.

Sure. Flat priors essentially allows for more unusual parameter observations to manifest in the likelihood and then posterior. Flat priors help keep these outliers in “check”, using knowledge of patterns observed in the past.

We must remember, in the likelihood, we are putting some faith in the parameter values that were collected. So if these are biased, a flat prior cannot make up for it.

For situations where we have little data but want a reasonable posterior, we might want a more informational prior, rather than a flat one.

Research done from: “Priors” A Student’s Guide to Bayesian Statistics, by Ben Lambert, SAGE, 2018.



All proofs are mine, unless indicated.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store