Write out the equations of operation for the network in Figure E14.3. Assume that all weights have a value of 0.5, and that all biases have a value of 0.
i. Assume that the initial network output is a(0) = 0.5 , and that the initial network input is p(1) = 1 . Solve for a(1).
ii. Describe any problems that you would have in simulating this network. Will you be able to apply the BPTT and RTRL algorithms to compute the gradients for this network? What test should you apply to recurrent networks to determine if they can be simulated and trained?
