Basic Bayesian Inference for MCMC Techniques: Exercises (Part 2)

In the first part , we saw a small introduction of the Bayesian inference and a first approach of Monte-Carlo techniques. Now, we will get through the Monte Carlo in order to obtain a random sample from the posterior distribution using some common techniques. Then, the next post will present the well-known Metropolis(-Hastings), Gibbs sampler, etc…

First, we will explore the Inverse transform sampling, which looks at computing the random sample relying on the inverse of the cumulative distribution function.
The results of this set of exercises can be found here.

Exercise 1: Inverse Transform Sampling

Relying on the Inverse transform sampling , generate a random sample from the following distributions and check by plot that the sample indeed follows that distribution :
a. Exponential distribution
b. Logistic distribution
c. Cauchy distribution where location and scale parameters are 0 and 1 respectively (also named the standard Cauchy distribution. Then, show that the mean (or any moments) are undefined for this.

Exercise 2: Rejection Sampling

We will use the rejection sampling algorithm to simulate data from an unknown distribution T, relying on another distribution that covers the support of the unknown distribution, and then accept/reject the samples. We want to estimate a beta distribution with shape parameters 2 and 5. Make your results reproductible.
a. Represent the density we want to estimate.
b. To do that, generate a large number of uniformly distributed samples on [0,1].
c. Compute and store the corresponding beta density values of these samples and the maximum of these values.
Use the maximum as the scaling factor (to ensure that T is not greater than the proposal density).
e. Show visually that the density is well approximated by the generated samples and the rejection sampling works.

Learn more about the concepts behind bayes in the online course Learning Data Mining with R. In this course you will learn how to:

• Work through a start to finish analysis process, including data mining
• Know how to compare regression models and the background of a Bayesian classifier
• And much more

Exercise 3: Slice Sampler

The slice sampling algorithm, although not the most efficient, is popular thanks to its
geometric interpretation. Following this algorithm, we want to estimate the same beta distribution with shape parameters 2 and 5.
a. Define the density function. Initialize the limits for the support of the distribution and the starting values.
b. Run the main loop that will generate the samples of interest.
c. Show visually that the generated sample has the expected density. You can also display a (subset of) the sample points on the plot and show that they indeed lie inside the density.
d. Show that this algorithm is less efficient in terms of computation time than the rejection sampling. Use the microbenchmark package. (Note that you could make this algorithm more efficient using vectorized solutions for example from the apply family).

Related exercise sets:

Spatial Data Analysis: Introduction to Raster Processing (Part 1) Spatial Data Analysis: Introduction to Raster Processing: Part-3 Density-Based Clustering Exercises Hacking statistics or: How I Learned to Stop Worrying About Calculus and Love Stats Exercises (Part-7) Big Data Manipulation in R Exercises