=========================================================================== CSC 363 Lecture Summary for Week 13 Fall 2008 =========================================================================== ----------------------------- Provably intractable problems ----------------------------- We've concentrated on P, NP, and PSPACE because they are all based on polynomial resource bounds, and our interest was in efficient computation. More importantly, a vast majority of "real-life" problems that arise naturally from various application domains belong to NP. We have seen how to prove problems are NP-complete (and examples of PSPACE-complete problems) and why this is evidence that these problems have no efficient solution. But are there problems that can be proved to have no efficient solution unconditionally? Definitions: EXP = U TIME(2^{n^k}) = { languages decided by TMs in time O(2^{n^k}) for some k } NEXP = U NTIME(2^{n^k}) = { languages decided by NTMs in time O(2^{n^k}) for some k } EXPSPACE = U SPACE(2^{n^k}) = { languages decided by TMs in space O(2^{n^k}) for some k } By Savitch's Theorem, NEXPSPACE = EXPSPACE. Known: L (_ NL (_ P (_ NP (_ PSPACE (_ EXP (_ NEXP (_ EXPSPACE Unknown: L ?= NL, NL ?= P, P ?= NP, NP ?= PSPACE, PSPACE ?= EXP, EXP ?= NEXP, NEXP ?= EXPSPACE In other words, we don't know how to prove that nondeterminism makes a difference, and we don't know how to prove that space is more powerful than time. Known: NL != PSPACE, P != EXP, NP != NEXP, PSPACE != EXPSPACE In other words, we can prove that exponential gaps make a difference (when measuring the same resource bound). How do we know these results? Because of so-called "hierarchy theorems": for most functions f_1, f_2 such that f_1 in o(f_2) (i.e., lim_{n -> oo} f_1(n)/f_2(n) = 0), TIME(f_1) is a proper subset of TIME(f_2) and SPACE(f_1) is a proper subset of SPACE(f_2). Idea: construct a decider D that runs in time (or space) O(f_2(n)) and behaves as follows: on input , D simulates M on input w for at most f_1(n) steps (or using at most f_1(n) space), and accepts if M rejects, rejects if M accepts (or exceeds the time or space bound). Then, L(D) in TIME(f_2) (or SPACE(f_2)) but L(D) must be different from every language in TIME(f_1) (or SPACE(f_1)), by construction. (See section 9.1 in textbook for technical details required to make this work.) Problems that are complete for EXP are known to be not in P, e.g., "generalized chess", "generalized checkers". Similarly, problems that are complete for EXPSPACE require more than polynomial space and are highly intractable, e.g., "inequivalence of regular expressions with squaring", "equivalence of regular expressions with exponentiation". But it doesn't stop there! We can define k-EXP, k-NEXP, k-EXPSPACE (deterministic time, nondeterministic time, space) 2^(2^...(2^n^t)...), where there are k exponentiations (so EXP = 1-EXP), and show there are languages in each class that are not contained in smaller classes. Even more, ELEMENTARY = 1-EXP U 2-EXP U ... = 1-EXPSPACE U 2-EXPSPACE U ... (since k-EXPSPACE is a subset of (k+1)-EXP). Problems complete for ELEMENTARY are decidable, but with such astronomical time or space bounds that they are completely intractable. Yet, "inequivalence of regular expressions with union, concatenation, and negation" requires running time 2^(2^(...2^(2^n)...)) where there are at least log n many exponentiations, so it's outside even ELEMENTARY -- yet still decidable! =========================================================================== NOTE: The following material was not covered in lecture; it is included here for your information. -------------- Back to PSPACE -------------- Inside PSPACE: Hierarchy of problems with increasing complexity, called the "polynomial hierarchy", with P = \Sigma_p^0, NP = \Sigma_p^1, \Sigma_p^2 that contains DP (including NP u coNP), \Sigma_p^3, ... Inside NP: If P != NP, then there are problems in NP that are neither in P nor NPc, and there are infinitely many intermediate classes between P and NPc, with complexity that gets larger and larger. For example, the following language: GRAPH-ISOMORPHISM = { : G and H are two undirected graphs that are isomorphic, i.e., there is a one-to-one and unto function f that maps vertices of G to vertices of H such that all corresponding edges are the same -- (u,v) is an edge of G iff (f(u),f(v)) is an edge of H } This is clearly in NP (a certificate is the function f, described as a list of pairs), but it is not known (or believed) to be in P, and it is not known (or believed) to be NP-complete. ---------------------------- Dealing with NP-completeness ---------------------------- We know NP-complete problems do not have efficient solutions (unless P = NP) but this doesn't make real-life application go away. NP-complete means there is no exact, efficient algorithm, based on worst-case analysis: in practical applications, worst-case may not come up often (if at all). Average-case performance may be more indicative (but much harder to compute properly) -- however, beware: uniformly random inputs may not be representative of "real life" inputs, and algorithm behaviour on uniformly random inputs may be misleading. Alternatively, sometimes possible to work with restricted classes of inputs. For example, 2SAT (- P, UNARY-SUBSET-SUM (- P (and so is the problem if all input integers have a value bounded by some polynomial function of the input size), etc. - Heuristics: compromise on efficiency -- some problems have algorithms that run in exponential time in worst-case but where worst-case does not seem to happen often in practice. Requires extensive testing on "real" inputs. Sometimes used even for problems in P (e.g., simplex). - Approximation: compromise on exactness -- find efficient algorithm that may not return exact answer but something "close", e.g., instead of finding a k-clique, maybe will find a (k/2)-clique or k vertices that are "almost" a clique. - Randomization: by making random choices, algorithm hopefully avoids worst-case behaviour. Intuitively, works well when many choices are reasonably good but it's hard to select deterministically for all inputs (e.g., QuickSort, primality testing). =========================================================================== REVIEW: Main topics covered - Computability . models of computation; robustness . diagonalization; countability/uncountability . decidability/recognizability; dovetailing . A_TM; undecidability/unrecognizability . mapping reductions (<=m); examples - Complexity . models of computation; P, NP, coNP . polytime reductions (<=p); Cook's theorem; NP-completeness . polytime self-reducibility . space complexity; PSPACE, L, NL; intractable problems - Reducibility (A <= B) is the central tool used. Understand it well!