==================================== WEEK 8 =============================== Shortest Paths -------------- * Recall that a "path" from u to v in a graph G is a sequence of edges (v_0,v_1), (v_1,v_2), ..., (v_{k-1},v_k) such that u = v_0, v = v_k. In a "simple" path, there is no repeated vertex (or edge). * In a "weighted" graph G, each edge e has an associated "weight" w(e), a real number (sometimes also called the "cost" of e and denoted c(e)). The weight of a path e_1,...,e_k is simply defined as the sum of the weights of the edges on the path = w(e_1) + ... + w(e_k). * We are interested in the problem of finding "shortest" paths, i.e., paths of minimum weight (or cost) between some vertices. We will look at two versions of this problem: Single-Source Shortest Paths and All-Pairs Shortest Paths. Single-Source Shortest Paths ---------------------------- * Problem: given a "source" vertex `s' in some weighted graph G, find the shortest paths from s to every other vertex in the graph. (Obvious applications in message routing on networks, etc.) * How are we going to solve this? Here's an idea: suppose that we already have some subset S of vertices such that for every vertex v in S, we know the shortest paths from s to v. Consider the vertices outside of S. What would be the shortest way to get to a from s? It might seem like simply adding a, b, c to S directly is the best thing to do, but it doesn't quite work. For example, there might be a shorter path to a going through c... What we do know is that the shortest path from s to a is *no longer* than the path we've found from vertices in S. This allows us to keep track of an upper bound on the length of the shortest paths from s to every vertex outside of S, but we don't know for sure if that upper bound is the exact value or not... So how do we get around this? Well, is there at least one vertex for which we know for sure we've found a shortest path? If we pick a vertex w outside of S that has the smallest upper bound, we must have in fact a shortest path from s to w because if there was a shorter way, we would have picked it first. Single-Source Shortest Paths ---------------------------- * [Review of idea behind Dijkstra's algorithm...] * This gives the following idea for an algorithm: for each vertex v in G, we will keep track of d[v], an upper bound on the weight of a shortest path from s to v. Initially, we set S = {}, d[s] = 0, and d[v] = oo for every v =/= s. Then, we reapeatedly pick a w outside of S such that d[w] is smallest, add it to S, and update d[v] for every v that is adjacent to w, until all the vertices are in S. If we also keep track of the predecessor for each vertex as we do this, we can reconstruct the paths from s to every other vertex easily. * This is known as Dijkstra's Algorithm: DIJKSTRA ( V, E, s ) for each v in V d[v] := oo; ("infinity") pred[v] := NULL; end for d[s] := 0; S := {}; V' := V; while ( V' is not empty ) do find a vertex u in V' such that d[u] is minimum; V' := V' - {u}; S := S U {u}; for each edge e = (u,v) in E if ( v is not in S ) and ( d[v] > d[u] + w(u,v) ) then d[v] := d[u] + w(u,v); pred[v] := u; end if end for end while END * Example: a -------------> c 3 _ 7 _ 0 _/ _/ \_ _/ _/ \/ 4 _/ s _/ e _/ _ \_ _/ _/ \/ _/ _/ 5 1 4 b -------------> d Tracing the algorithm on this graph produces the following values for S, d[], and pred[] after each iteration of the while-loop (for every iteration, we indicate the values of d[] and pred[] only for the vertices that remain outside of S). S d: s a b c d e pred: s a b c d e {} 0 oo oo oo oo oo - - - - - - {s} 3 5 oo oo oo s s - - - {s,a} 5 10 oo oo s a - - {s,a,b} 9 6 oo b b - {s,a,b,d} 9 10 b d {s,a,b,d,c} 9 c {s,a,b,d,c,e} Which gives the following final values (from which we can easily find shortest paths and their lengths). S d: s a b c d e pred: s a b c d e {s,a,b,d,c,e} 0 3 5 9 6 9 - s s b b c All-Pairs Shortest Paths ------------------------ * Suppose we want to find the shortest paths between *every* pair of vertices u,v in G. One possible way to do this would be to simply run Dijkstra's algorithm once for each each vertex as the source, but this seems a bit inefficient. * How else can we do this? The idea is to start with the distances between each pair of vertices equal to the length of the edge between them (if there is one), or to "infinity" (if there is no edge), and then to update this distance little by little, by considering paths that use more and more intermediary vertices. * Why are we doing things this way? This works because of the following property: if we take a shortest path from u to v that goes through a vertex w, then it *must* be the case that the part of the path from u to w is a shortest path from u to w, and similarly from w to v (otherwise, we could replace that part of the path and end up with something shorter from u to v). * More formally, for each pair i and j of vertices, we will keep track of a "distance" D[i,j] in the following way: D[i,j] = length of a shortest path from i to j that uses only vertices from {0,...,k} between i and j; Note that we are *not* saying that the path must use *all* of the vertices in {0,...,k}, or that it must use them in any specific order; all we're saying is that a path from i to j should only be considered if it does *not* use any of the vertices {k+1,...,n-1} (where the graph has n vertices). * So how will we compute the values of D? We need initial conditions, a "base case" when we don't consider paths using *any* intermediate vertices: { 0 if i == j, D[i,j] = { w(i,j) if i != j and (i,j) is in E, { oo otherwise. Then, we can compute the values of D[i,j] using more and more intermediary vertices, one at a time. Once we know what it is we want to compute, the way to compute it is actually not too hard to figure out: consider a shortest path from i to j that uses only vertices from {0,...,k} between i and j. Either vertex k appears on that path or it doesn't. If it doesn't, then this is in fact a shortest path from i to j that uses only vertices from {0,...,k-1} between i and j, so D[i,j] doesn't change. If vertex k appears on the path, then consider the part of the path from i to k: this must be a shortest path from i to k that uses only vertices from {0,...,k-1} between i and k (and the same holds for the part of the path from k to j), so we must have D[i,j] = D[i,k] + D[k,j]. Since we don't know ahead of time whether or not k appears on a shortest path from i to j, we simply check both possibilities and pick the smallest. * This gives us the following algorithm, known as Floyd-Warshall's algorithm: FLOYD-WARSHALL( V, E ) for i := 0 to n-1 do for j := 0 to n-1 do if i == j then D[i,j] := 0; else if (i,j) is in E then D[i,j] := w(i,j); else D[i,j] := oo; end if end for end for for k := 0 to n-1 do for i := 0 to n-1 do for j := 0 to n-1 do if D[i,k] + D[k,j] < D[i,j] then D[i,j] = D[i,k] + D[k,j]; end if end for end for end for END * Note that this simply computes the shortest distance between every pair of vertices, and not the actual paths: we need to do a bit of extra work to get the paths... * Why does Floyd-Warshall's algorithm work? What if I find a short route from i to j using 0 and then I find a shorter way from i to 0 through 1 (so that I go i -> 1 -> 0 -> j): how does the way from i to j change? The algorithm will find the shorter way because it will find the short way from 1 to j through 0 when it is doing the update using 0, and then the way from i to j will be updated through 1 (trace it on a small example and you'll see). * Also, how do we know that the values are not getting overwritten with wrong information since we constantly change them? When we are computing the values of D[i,j], they can only change depending on the values of D[i,k] and D[k,j] (i.e., the values stored in row k and column k of the array), and those values themselves do *not* change (since D[k,k] = 0). * Let's trace Floyd-Warshall's algorithm on the following undirected graph: 1 (1)-----(2) | / | 4 | 1/ | 3 | / | (0)-----(3) 1 D_{-1}| 0 1 2 3 D_0| 0 1 2 3 D_1| 0 1 2 3 ---+------------ ---+------------ ---+------------ 0 | 0 4 1 1 0 | 0 4 1 1 0 | 0 4 1 1 | | | 1 | 0 1 oo 1 | 0 1 5 1 | 0 1 5 | | | 2 | 0 3 2 | 0 2 2 | 0 2 | | | 3 | 0 3 | 0 3 | 0 D_2| 0 1 2 3 D_3| 0 1 2 3 ---+------------ ---+------------ 0 | 0 2 1 1 0 | 0 2 1 1 | | 1 | 0 1 3 1 | 0 1 3 | | 2 | 0 2 2 | 0 2 | | 3 | 0 3 | 0 * Note that this algorithm does not directly give us the actual paths between each pair of vertices. In order to get them, we can keep track of extra information: each time that we update D[i,j] for some k because we've found a shorter path, remember that the intermediate node used was `k' in a second array M[i,j]. Then, at the end, we can use this array to reconstruct the paths.