This is a two-hour exam. The exam is open notes and open any electronic reading device, provided they are used solely for reading material already stored on them and not for any other form of communication or information retrieval. Calculators are permitted though not needed. Begin each problem on a new page in the answer book. Good luck!
1. Short questions
There are 6 short questions. For each question, you should give your answer followed by a brief
justi cation.
a. Suppose F(x) is the cumulative distribution function (CDF) of a continuous
random variable X. What is the variance of F(X)?
b. Compare E[XjY 3] with E[XjY ]. (Choose \ ", \", \=" or \not comparable").
c. Compare the MSE in optimal estimation of X based on Y with the MSE in optimal estimation of X based on Y 2.
d. Suppose X is a random variable and MX(t) = E[etX] exists for all t 2 R. For t < 0, compare P(X ) with e tMX(t).
e. is a nn matrix with entries ij = minfi; jg. Prove this is a covariance matrix by describing a construction of an n dimensional random vector with covariance matrix .
Hint: Start with n iid zero mean, unit variance random variables .
f. is a n n matrix with entries ii = 1 and ij = for 2 (0; 1) and i 6= j. Prove is a covariance matrix by describing a construction of an n dimensional random vector with covariance matrix .
Hint: Start with n + 1 iid zero mean, unit variance random variables.
2. A signal X 2 f0; 1g is transmitted through a binary symmetric channel with crossover proba- bility p. Assume X = 0 or 1 with equal probability. Suppose X is sent n times, each time the channel output Yi follows Yi = X Zi; where \" is the modulo 2 addition, and Zi's are independent Bern(p) random variables. Further assume X and all Zi's are independent.
a. Compute
P(Y1 = y1; ::; Yn = ynjX = x)
where yi 2 f0; 1g for i = 1; 2; ::; n.
b. (6 points) Suppose T =
Pn
i=1 Yi. Compute
P(T = t) for t = 0; ::; n
c. Compute the conditional distribution fY1; ::; Yng given T, i.e.
P(Y1 = y1; ::; Yn = ynjT = t)
Further show that fY1; ::; Yng is conditionally independent of X given T.
d. Find the decoder ^X
that minimizes P(X 6= ^X
). (Hint: the answer can be given
in terms of T)
3. X N(0; P) is sent repetitively n times through an additive Gaussian channel. Each time the channel outputs Yi = X + Zi, where Zi N(0; 1). Suppose Zi's and X are all independent.
a. What is the covariance matrix of [X; Y1; ::; Yn]?
b. Compute the MMSE estimator of X given the rst observation Y1. What is the
mean square error?
c. Compute the MMSE estimator of X given Y1; ::; Yn. What is the mean square
error?