=========================================================================== CSC 363 Lecture Summary for Week 7 Fall 2009 =========================================================================== ------------------------ Computational Complexity ------------------------ Outline (topics and textbook sections): - BRIEF review of complexity analysis and asymptotic notation, complexity classes TIME(t(n)), robustness (7.1). - The classes P and NP -- definitions and examples (7.2, 7.3). - Polytime reducibility, self-reducibility; NP-completeness (7.4, 7.5). - Space complexity; other complexity classes (8.2, 8.2, 9.1). - Dealing with NP-completeness (10.1, 10.2). ------------------- Complexity analysis ------------------- Motivation: answer question "what is *efficient* computation?" (by analogy with first part of course that considered "what is computation"). First, must agree on how to measure efficiency. Two standard measures: time and space. Easy to define on TM model: time = number of steps in computation = number of transitions executed; space = number of tape cells. Worst-case vs. average-case measures as a function of input size are defined as usual, using asymptotic notation (big-Oh, \Omega, \Theta). Complexity classes TIME(t(n)), SPACE(t(n)), for any function t : N -> R^+: TIME(t(n)) = { L : L is a language decided by some TM that runs in worst-case time O(t(n)) } SPACE(t(n)) = { L : L is a language decided by some TM that runs in worst-case space O(t(n)) } For example, TIME(n^2) contains every language decided by a TM in worst-case time O(n^2). Example: A = { 0^n 1^n | n >= 0 }. Can be decided in time O(n^2) by repeatedly scanning back-and-forth, crossing off a single 0 and 1 during each pass. But possible to do better by repeatedly crossing off half of the 0s and 1s until none are left, rejecting if at any point the number of 0s and 1s are not both even or odd -- this takes time O(n log n) only. This time cannot be improved: possible to show any language decided in time o(n log n) on standard TM is regular! With two tapes, A can be decided in time O(n): go over all of the 0s and copy them to second tape, then when first head starts going over 1s move backwards over 0s on second tape. If the end of the 1s and 0s is reached at the same time, accept; otherwise, reject. Note: A (- SPACE(n) no matter which model is used, as both TMs use only a linear amount of space (proportional to the length of the input). So specific model used changes meaning of complexity classes! ----------------- Computation model ----------------- Multitape TM: Every multitape TM that runs in time t(n) >= n has an equivalent single-tape TM that runs in time O((t(n))^2). Convert multitape TM M to single-tape TM S. Each step of M requires two passes over entire tape contents of S. Since M runs in time t(n), each tape of M contains at most O(t(n)) symbols so tape of S contains at most O(k * t(n)) = O(t(n)) symbols (remember number of tapes k is constant); hence, each step of M requires time O(t(n)) on S. Entire computation requires time O(t(n)) * O(t(n)) = O((t(n))^2). Note: Space unaffected -- space required to simulate multi-tape TM is within a constant factor (in fact, only an additive constant) more. Non-deterministic TM: Every NTM that runs in time t(n) >= n has an equivalent single-tape TM that runs in time 2^O(t(n)). Convert NTM M to DTM D. If M is a decider, then every branch halts. Since running time is defined as height of computation tree (length of longest computation branch) and M runs in time t(n), there are at most b^{t(n)} leaves in computation tree (where b = maximum branching factor). Performing BFS on this tree can be done within time O(t(n) * b^{t(n)}) = O(C^{t(n)}) (for some C >= b) = 2^O(t(n)). Other variants: minor modifications of basic model or ones above don't affect time complexity significantly (e.g., two-way infinite tape can be simulated using 2 tapes with no loss of efficiency). Consequence: TIME(t(n)) sensitive to particular model. Not as nice as for computability, where model did not affect outcome. Can we do better? "Tractability thesis": All reasonable *deterministic* models are polynomially equivalent (i.e., can simulate one another with at most a polynomial factor loss of efficiency). This includes models not based on TMs, e.g., register machines, lambda calculus, etc. The same holds among all reasonable non-deterministic models. ----------- The class P ----------- Concentrate on "coarse scale": ignore polynomial differences. This allows (almost) model-independent characterization (except for nondeterminism). P = U_{k >= 0} TIME(n^k) = { L (_ \Sigma* : L can be decided by some deterministic algorithm (TM or otherwise) in worst-case polytime } Importance: - Robust: does not depend on details of model, as long as it is deterministic. - Useful: captures rough notion of efficiency. Q: What about real-life need to tell difference between runtimes like n^3 and n^2? A: Situation similar to O notation: deal with large differences first, then if needed, work with one specific model to study more specific differences. Conventions: - Describe computation in stages, where each stage is "obviously" computable in polytime (with justification) and at most polynomially many stages are executed, on any input (with justification). - Encodings: "reasonable" now means "can be encoded/decoded in polytime". In particular, numbers must be represented in some base >= 2 (unary notation requires k symbols to represent integer k, exponentially worse than the log_b(k) symbols required in base b). Examples: Almost all algorithms you've seen in other courses. - PATH = { : graph G contains a path between nodes u, v } Trying all possible paths in G requires worst-case exponential time. However, DFS (depth-first search) or BFS (breadth-first search) solve problem in polytime. ------------ The class NP ------------ NTIME(t(n)) = { L (_ \Sigma* : L can be decided by a NTM in worst-case time O(t(n)) } NP = U_{k >= 0} NTIME(n^k) = { L (_ \Sigma* : L can be decided by some non-deterministic algorithm in worst-case polytime } By tractability thesis, NP independent of specific details of nondeterministic model (as long as it's nondeterministic). Nondeterminism and Verifiers: - Recall acceptance condition for NTMs: there exists an accepting computation path in computation tree. - Verifiers capture that notion directly: verifier for language L = algorithm V that takes input such that x (- L iff -] c such that V accepts . Runtime measured as a function of n = |x| only (ignoring c). Note: c is called a "certificate" -- a piece of information that makes it possible to "verify" that x has a certain property efficiently. - Every polytime verifier can be simulated by polytime NTM (use non-determinism to check all possible values of certificate c), and every polytime NTM can be simulated by polytime verifier (use certificate c to select computation path to take), so both models are equivalent. Example: COMPOSITES = { x : x is a composite number } COMPOSITES (- P? Unclear (checking all possible factors not polytime because this could require checking up to sqrt{x} possibilities; as a function of n = |x| = log_2 x (in binary), this represents sqrt{x} = sqrt{2^n} = 2^{n/2} possibilities: exponential). COMPOSITES (- NP? Yes: On input : 1. Check that 1 < c < x (interpreting c as integer). 2. Check that c divides x evenly. Accept if all checks succeed; reject if any fail. If x is composite, then verifier above will accept when c is a factor of x. If x is prime, then verifier will reject for all values of c. Moreover, verifier runs in polytime: comparing integers (1 < c < x) can be done in linear time; dividing integers can be done in quadratic time (remember we measure time as a function of input size = number of bits to write inputs). Note: Don't think of non-deterministic algorithms in the same way you think of deterministic ones. Non-deterministic algorithms are never meant to be "run" on an input to "see what the algorithm does", because they do not have a single behaviour (the computation is a tree) -- rather, a non-deterministic algorithm is simply a way to describe the acceptance condition for strings in the language, abstractly. But what about actually deciding languages in NP? How does this relate to verifiers? Languages in NP can all be decided by algorithm of the form: Generate all candidates c as follows: [...] For each candidate c, [...] where second part always takes polytime, but first part may take longer (e.g., because there are too many candidates). Notion of verifier is a way to write second stage of such a decider directly. Given that nondeterminism is not "practical", why use it? Turns out a large number of real-life problems have no known efficient solution (i.e., are not known to belong to P), yet can be solved efficiently using nondeterminism. So nondeterminism allows us to characterize a large class of problems. Also, nondeterminism is an elegant way to add (what seems to be) significant power to the model.