The Woodbury Formula


NOTE: I will be using MATLAB notation throughout for convinience, so that 'A'' means "A transpose", 'inv(A)' means "A inverse" and 'eye(n)' means " I_n", the n x n identity matrix.

In tutorial today I talked about the Sherman-Morrison formula, which allows one to efficiently compute x = inv(A_bar)*b (i.e. the solution of the linear system A_bar*x = b) given an LU factorization of A, where A_bar = A - M for some rank 1 matrix M (so that M can be expressed as u*v' for some two column vectors u and v). The book states the formula as x = inv(A)*b + inv(A)*u*inv(1 - v'*inv(A)*u)*v'*inv(A)*b. But this is a pretty funny way of writing it, because c = inv(1 - v'*inv(A)*u)*v'*inv(A)*b = (v'*inv(A)*b)/(1 - v'*inv(A)*u) is just a scalar, which means that the formula can be restated as x = inv(A)*b + c*inv(A)*u (because c is a scalar and inv(A)*u a vector, we have c*inv(A)*u = inv(A)*u*c), where c is as above (see line 3 of algorithm 2.6 on p. 82). The book stated the Sherman-Morrison formula in such a convoluted way to emphasize the connection to the Woodbury Formula (which I didn't cover in tutorial), a much more general formula that applies for any rank k modification (and reduces to the Sherman-Morrison formula in the special case k = 1). For the Woodbury formula, the setup is A_bar = A - M, where rank(M) = k, M = U*V' (U and V are now n x k matrices) and x = inv(A)*b + inv(A)*U*inv(eye(n) - V'*inv(A)*U)*V'*inv(A)*b. Because in this case U and V are matrices and not vectors, the order of multiplication does matter (recall that matrix multiplication is not commutative in general).