Question: Let z = trace (AT CA). Show that the derivative of z with respect to the matrix A is
∂z/∂A = CA + CTA
Therefore, if C = Q is symmetric, then the derivative is 2QA.
We have restricted the discussion here to real matrices and vectors. It often happens that we want to optimize a real quantity with respect to a complex vector. We can rewrite such quantities in terms of the real and imaginary parts of the complex values involved, to reduce everything to the real case just considered. For example, let Q be a Hermitian matrix; then the quadratic form k†Qk is real, for any complex vector k. As we saw in Exercise, we can write the quadratic form entirely in terms of real matrices and vectors.
Exercise: Show that if z = (z1, ..., zN)T is a column vector with complex entries and H = H† is an N by N Hermitian matrix with complex entries then the quadratic form z†Hz is a real number. Show that the quadratic form z†Hz can be calculated using only real numbers. Let z = x + iy, with x and y real vectors and let H = A+iB, where A and B are real matrices. Then show that AT = A, BT = -B, xTBx = 0 and finally,
Use the fact that z†Hz is real for every vector z to conclude that the eigenvalues of H are real.