Question

Linear Algebra (Cont’d)

3. Let A = 1-X(XTX)-"XT. (a) Must A be square? (b) Must X' X be square? (e) Must X be square?

Answer 1

3. Given $\dpi{80} A = I - X(X^TX)^{-1}X^T$ . We put the respective dimensions aslo, so as to clear up the equation, ie $\dpi{80} A = \underset{m*m}{I} - \underset{m*n}{X}(\underset{n*m}{X}^T\underset{m*n}{X})^{-1}\underset{n*m}{X}^T$ . Supposing m and n be positive integers, the identitty matrix will necessarily have equal rows and columns, and supposing X is not a square matrix, having dimesntions m*n, the transpose of X would heve the opposite dimensions, ie n*m.

(a) If X was squared, then A would obviously be squared. Supposing X being a non-square matrix, we have $\dpi{80} A = \underset{m*m}{I} - \underset{m*n}{X}(\underset{n*m}{X}^T\underset{m*n}{X})^{-1}\underset{n*m}{X}^T$ or $\dpi{80} A = \underset{m*m}{I} - \underset{m*n}{X}\underset{n*n}{(X^TX)}^{-1}\underset{n*m}{X}^T$ (as matrix multiplication of n*m and m*n matrix would produce n*n matrix) or $\dpi{80} A = \underset{m*m}{I} - \underset{m*m}{X(X^TX)^{-1}X^T}$ (as matrix multiplication of m*n and n*n and n*m would produce matrix of m*m). Hence, matrix A must be squared, even if m is not equal to n, ie even if X is non-squared.

Note: Inverse is only possible for squared matrix, and the dimesions stays the same after the inversion.

(b) If X was squared, then $\dpi{80} X^TX$ would be squared as well. But if X have dimesntions m*n, where m is not equal to n, then $\dpi{80} X^T$'s dimension would be the opposite, ie n*m. Hence, $\dpi{80} (\underset{n*m}{X}^T\underset{m*n}{X}) = \underset{n*n}{(X^TX)}$ , ie even if X is not squared, $\dpi{80} X^TX$ must necessarily be squared.

(c) From above calculations, X must not be necessarily squared to satisfy equation $\dpi{80} A = I - X(X^TX)^{-1}X^T$ .