Consider the learning problem in Example, where the input space is X = [-1, +1], the target function is f(x) = sin(πx) , and the input probability distribution is uniform on X. Assume that the training set V has only two data points (picked independently), and that the learning algorithm picks the hypothesis that minimizes the in sample mean squared error. In this problem, we will dig deeper into this case.
For each of the following learning models, find (analytically or numerically)
(i) the best hypothesis that a p proximates f in the mean squared error sense (assume that f is known for this part),
(ii) the expected value (with respect to D) of the hypothesis that the learning algorithm produces, and
(iii) the expected out of sample error and its bias and var com ponents.
(a) The learning model consists of all hypotheses of the form h(x) = ax + b (if you need to deal with the infinitesimal probability case of two identical data points, choose the hypothesis tangential to f).
(b) The learning model consists of all hypotheses of the form h(x) = ax. This case was not covered in Example.
(c) The learning model consists of all hypotheses of the form h(x) = b.
Example
Consider a target function f(x) = sin(nx) and a data set of size N = 2. We sample x uniformly in [-1, 1) to generate a data set (x1, Y1), (x2, Y2); and fit the data using one of two models:
H0: Set of all lines of the form h(x) = b;
H1: Set of all lines of the form h(x) = ax + b.
