Response to the following:
Show that, in Example 1:, the weights 1/4 and 3/4 are irrelevant for the maximization of Q(θ'|θ, x).
Example 1:
Using the normal mixture likelihood of Examples 2, we saw in Figures that the likelihood is bimodal when associated with a sample from the normal mixture. Using the missing data structure exhibited in Example 2 leads to an objective function equal to
n
Q(θ’|θ,x) = -1/2 ∑ Eθ[Zi(xi - μ1)2 + (1 - Zi)(xi - μ2)2 |X]
i=1
Example 2:
The likelihood associated with the mixture model
1/4 N (μ1,1)+3/4N(μ2,1)
is bimodal, as seen in Figure for a simulated sample of 400 observations from this mixture with μ1 = 0 and μ2 = 2.5, actually produced by
> da=rbind(rnorm(10ˆ2), 2.5+rnorm(3*10ˆ2))
> like=function(mu){
+ sum (log((.25*dnorm(da-mu[1]+.75*dnorm(da-mu[2]))))}
and by applying the R function contour to a grid of points where the loglikelihood function like is computed. When using nlm, the modes are obtained within a few iterations, depending on the starting points, and the intermediate values of the Newton-Raphson sequence can be plotted by
> sta=c(1,1)
> mmu=sta
> for (i in 1:(nlm(like,sta)$it))
+ mmu=rbind(mmu,nlm(like,sta,iter=i)$est)
> lines(mmu,pch=19,1wd=2)
where the function like has been redefined as its inverse to account for the fact that nlm produces a local minimum. Note that some starting points produce warnings:
NA/Inf replaced by maximum positive value in: nlm(like, sta)
meaning that the (numerical approximation to the) Hessian is not invertible at the current value. The sequences represented in Figure all end up in one of the two modes, but with highly nonlinear patterns. For instance, the starting point (- 1, -1) corresponds to a very steep gradient and thus bypasses the main mode (- 0.68, 1.98) to end up at the secondary one (lower in likelihood). Although all represented sequences do converge, starting farther away from the modes may produce divergent sequences.
