Derive jeffrys prior for this model derive the posterior

Question

Question 1. The table schedules Year below gives the number of fatal accidents and deaths on airline flights per year over a ten-year period.

Year	Accidents
1976	24
1977	25
1978	31
1979	31
1980	22
1981	21
1982	26
1983	20
1984	16
1985	22

(a) Assume that the number of fatal accidents each year independently follow a Poisson(θ) distribution. Derive Jeffry's prior for this model. Derive the posterior distribution of θ under this prior?

(b) Obtain posterior samples from the model described in part (a). Provide the density plot of your samples as well as the 95% posterior credible interval and MAP estimate for θ|data.

(c) Now obtain samples from the posterior predictive to infer on the number of fatal accidents in 1986. Provide the density plot of your samples as well as the 95% posterior credible interval and MAP estimate for y^~|data.

(d) Assume now that the numbers of fatal accidents in each year t independently follows a Poisson(θ_t) where log(θ_t) = α + βt. Choose a reasonable noninformative prior for p(α, β). Write our the joint posterior for p(α, β|data) and formally write out a Metropolis algorithm that updates α and β together (be sure to be specific about the index of iterations).

(e) Implement your algorithm in (d). Provide discussion and plots regarding your tuning parameter(s), burn-in, autocorrelation, acceptance, and thinning. Obtain 2000 independent posterior samples from p(α, β|data) and plot the joint and marginal posterior densities. Obtain MAP and 95% credible intervals for the posterior rate of fatal accidents per year (i.e., θ_t|data) at each year: 1976 -1985. Discuss what happens to θ_t|data over time in context of the problem.

(f) Using your posterior samples of α and β to predict the number of fatal accidents in the year 1986. Provide the density plot of your predicted samples as well as the 95% posterior credible interval and MAP estimate. Discuss and compare these results to the results in (c). Which model seems more appropriate for these data? Defend your answer.

Question 2. The data file hearing.txt is from an experiment to calibrate word lists used to measure the hearing ability of subjects. The four word lists had been designed so that they should be equally difficult to perceive, but were designed for normal-hearing subjects in an environment without background noise. The data in this experiment were collected in the presence of a noisy background. Each column is a word list, and each row is a subject. The entry is their score on that list (each subject was tested on all four lists). We will consider a two-way ANOVA model such that we will assume a Normal likelihood for each with mean that depends on both the subject and the list. In other words we will consider both a subject effect (θ_h) as well as a list effect (θ_j). We will assume conjugate priors. The full hierarchical model is given by:

yh_j|θ_h, Φ_j (σ²) ~ N(θ_h + Φ_j, σ²)

θ_h|μ, σ² ~ N(μ, σ²)

θ_j|σ² ~ N(0, σ²/4)

μ|σ² ~ N(30, σ²/9)

σ² ~ Γ^-1(1, 1)

for h = 1,......n and j = 1,...... k with n = 24 and k = 4.

(a) Write out the joint likelihood, f(y|θ, Φ, σ²).

(b) Derive the full posterior conditional distribution for θ_h. That is find the form of f(θ_h|θ_-h, Φ, μ, σ², y)

(c) Derive the full posterior conditional distribution for Φ_j. That is find the form of f(Φ_j|Φ_-h, θ, μ, σ², y)

(d) Derive the full posterior conditional distributions for the hyperparameters: f(μ|Φ, θ, μ, σ², y) and f(σ²|Φ, θ, μ, σ², y)

(e) Fit the model with MCMC. Show your trace plots for μ for at least three θ_h's, and for at least two Φ_h's of your choice. Remove burn-in as appropriate. Be sure you obtain at least 2000 independent posterior samples.

(f) What are the maximum likelihood estimates of the Φ_h's? Make a plot comparing the MLE's to you estimated posterior means of the θ_h's.

Use the abline(0,1) to add the y = x line to you pot. Comment on what you see. How does this Bayesian analysis compare to a simple frequentist (mle) one?

(g) Of interest to the researchers is whether the lists have the same level of difficulty. Plot the densities of the posterior for all four θ_j's. Construct 95% credible intervals for each θ_j and see if they include zero. What can you conclude about the lists?

Question 3. Consider the Load.txt dataset which was collected from a study that examined the heating load and cooling load requirements of buildings (that is, energy efficiency) as a function of building parameters. The dataset contains eight (p = 8) attributes (or features, denoted by X1...X8) and two responses (or outcomes, denoted by y1 and y2). The aim is to use the eight features to predict each of the two responses. There are a total of n = 768 cases.

X1 | Relative Compactness

X2 | Surface Area

X3 | Wall Area

X4 | Roof Area

X5 | Overall Height

X6 | Orientation

X7 | Glazing Area

X8 | Glazing Area Distribution

Y1 | Heating Load

y2 | Cooling Load

Source: A. Tsanas, A. Xifara: Accurate quantitative estimation of energy perfo rmance of residential buildings using statistical machine learning tools, Energy and Buildings, Vol. 49, pp. 560-567, 2012

For this exam, you will explore which explanatory variables are important in predict¬ing the heating load and the cooling load via Bayesian lasso regression. Specifically, you will fit the following model:

y ~ N(1_nμ + Xβ, σ² I_nxn)

β|∑_o ~ N(0, σ²∑₀)

where ∑₀ = diag(τ₁², τ_p²)

T²|λ ~ Π^P_j=1 Exp(λ²/2) note that λ²/2 is the rate parameter

Assume the following priors: p(μ) ∝ 1, p(σ²) ∝ (σ²)^-1, λ² ~ Γ(0.01, 0.01). Provide a detailed analysis of lasso variable selection on these data.

(a) Fit the model above to the Load dataset using y1 as the response and X1 - X8 as explanatory variables. Summarize your results via plots/tables and discussion.

(b) Fit the model above to the Load dataset using y2 as the response and X1 - X8 as explanatory variables. Summarize your results via plots/tables and discussion.

(c) Compare the lasso results in (a) and (b)

4. Read carefully through Roderick Little's 2011 paper Calibrated Bayes, for Statistics in General, and Missing Data in Particular. Provide a detailed report (minimum 1 full page) of the issues and ideas presented in this paper. Summarize the pros and cons of the various imputation methods. What is you personal opinion on missing data imputations?

Article - Calibrated Bayes, for Statistics in General, and Missing Data in Particular by Roderick Little

https://www.dropbox.com/s/3mngxati2qr9gyy/Homework.zip?dl=0