===========================================================================
CSC 363                 Lecture Summary for Week 12             Winter 2008
===========================================================================

Recall "picture of the world": P (_ NP n coNP (_ PSPACE (_ decidable.

Given we can't prove P != PSPACE, what to do?  Same as for NP: identify
"hardest" problems in PSPACE.

Language A is PSPACE-complete if:
 -  A (- PSPACE.
 -  A is PSPACE-hard: B <=p A for all B (- PSPACE.

Notice we still use polytime reductions (<=p), because we're concerned with
P vs PSPACE so we need notion of reduction no stronger than smallest class
of interest, to ensure if B <=p A and A (- P, then B (- P.  In other words,
we can't use something like "polyspace reduction" because then A (- P and
B <=pspace A would *not* guarantee B (- P (only B (- PSPACE).

Examples:

 -  Quantified boolean formula = propositional formula with quantifiers on
    propositional variables, e.g.,
        \-/ p -] q ( ((p \/ q) -> r) /\ (~p -> (q /\ r)) ).
    Fully quantified boolean formula = all variables quantified and all
    quantifiers at the front ("prenex normal form"), e.g., formula above
    with "\-/ r" added in front.
    Fully quantified boolean formulas are always either true or false.
    Note: asking "is propositional formula F satisfiable" same as asking
    "is -] p_1 -] p_2 ... -] p_k F true" (where p_1,p_2,...,p_k are all
    variables of F).

    TQBF = { <F> : F is a true fully quantified boolean formula }

    TQBF (- PSPACE:
        On input F:
         -  If F has no quantifiers, then evaluate F and accept iff it is
            true.
         -  If F = -] x F', then recursively evaluate F'[x=0] and F'[x=1]
            and accept iff either computation accepts.
         -  If F = \-/ x F', then recursively evaluate F'[x=0] and F'[x=1]
            and accept iff both computations accept.
        Entire recursion tree covers exponentially many possible settings
        of variables, but tree is traversed in depth-first manner, so only
        need to store information for one path.
        Recursion depth = number of variables of F and each level stores
        value of one variable: total space used for recursion is linear.
        Evaluating F at bottom of each path also requires linear space, but
        this can be shared between calls.

    TQBF is PSPACE-hard:
        For any language A (- PSPACE, construct a quantified formula that
        represents the computation of a PSPACE TM for A.  Details are in
        the textbook.

 -  Many types of two-player games for which we can ask: given a certain
    game configuration, does player 1 have a guaranteed win?

    For example, "Generalized Geography": on directed graph G with selected
    start node, players take turns moving along edges, without ever going
    back to previously visited nodes.  Last player to move wins.

    GG = { <G,b> : G is a directed graph, b is a node in G, and player I
                   has a winning strategy playing generalized geography
                   from start node b, i.e., there are moves for player I to
                   win no matter how player II moves }

    GG (- PSPACE:
        On input <G,b>:
         -  Reject if no outgoing edge from b.
         -  Otherwise, remove b and all its edges to get G_1;
            let b_1, ..., b_k be endpoints of edges from b.
         -  Recursively run on inputs <G,b_1>, ..., <G,b_k>;
            reject if all accept (player II wins); accept if one rejects.
        Recursion depth = number of nodes in G (linear).  Information
        stored at each call = nodes visited so far; since each node stored
        at most once for each possible path in recursion tree, total
        information stored is linear.

    GG is PSPACE-hard:
        TQBF <=p GG: see construction in textbook.

Even more so than NP-complete problems, PSPACE-complete problems have no
time-efficient solutions.  However, they still have efficient solutions in
memory usage, and their time complexity has not been proven outside P.

So are there any decidable languages we can _prove_ outside P?  Before
considering this, one more important space complexity class.

---------
Log-space
---------

 -  L = SPACE(log n) = { languages decided by TM in space O(log n) }
    NL = NSPACE(log n) = { languages decided by NTM in space O(log n) }
    coNL = { complements of languages in NL }

 -  Q:  How can TM use less than linear space when it needs at least that
    much to store input?
    A:  Measure "work" space independently of space to store input: use
    2-tape TM with read-only "input tape" and read-write "work tape", and
    count only cells used on work tape.  (Similar to working with data on
    read-only DVD when main memory much smaller.)

 -  Note:  Sublinear time not useful (can't even read entire input) but
    sublinear space useful (we'll see examples).

 -  Example 1: { 0^k 1^k : k >= 0 }
     .  Can't just scan back-and-forth marking 0s and 1s because input tape
        is read-only and marking would require copying to work tape, using
        more than log n space.
     .  Idea: count instead!  Read over 0s and use work tape to record
        number of 0s read as a binary counter; when we start reading 1s,
        decrease counter; accept iff no 0 is encountered following a 1 and
        counter = 0 when we reach end of input and not before.
     .  Space usage: counting up to k requires O(log_2 k) bits, so space is
        logarithmic.

 -  Think of L as languages that can be recognized by using a fixed number
    of counters/pointers (counters can be used to keep track of positions
    into the input string).

 -  Example 2:
        A_DFA = { <A,w> : A is a DFA that accepts input string w }
    Given <A,w>, keep track of current state of A and current input symbol
    of w in order to simulate A on w: this can be done with two
    counters/pointers.

 -  Example 3:
        PATH = { <G,s,t> : G is a graph that contains a path from s to t}
    No known deterministic log-space algorithm, but easy nondeterministic
    log-space algorithm: store index of current node, start at s and
    nondeterministically select next node, accepting when t is reached (or
    rejecting after n steps, where n = number of nodes of G).
    This only requires room to store one node index, O(log n), and there is
    some computation path that accepts iff there is some path from s to t
    in G.

 -  L (_ NL but NL (_ L unknown: Savitch's Theorem shows only
    NL (_ SPACE((log n)^2).

 -  What about NL and P?
    PATH is NL-complete (w.r.t. L):
     .  PATH (- NL
     .  for all A (- NL, A <=L PATH, using "log space reduction" <=L
        (using 3-tape TM with read-only input tape, write-only output tape,
        and worktape, and measuring only worktape used)
    Idea:  The question "does w belong to A" is equivalent to "is there a
    path from the initial configuration to an accepting configuration in
    the computation tree of the nondeterministic log space TM for A".

 -  Note:  If A (- L and B <=L A, then B (- L.  However, must be careful:
    output of log space reduction could take up more than log space.  To
    get a log space algorithm for B, must use log space algorithm for A and
    recompute log space reduction each time, keeping at most logarithmic
    number of output symbol at a time on work tape.

 -  Since PATH (- P, and A <=L B implies A <=p B (TM running in space
    O(log n) has at most n * 2^O(log n) possible configurations = O(n^k)
    for some constant k), NL (_ P.

 -  L = NL?  Unknown!  P = NL?  Unknown!
    However:  NL = coNL!  NL != PSPACE!

-----------------------------
Provably intractable problems
-----------------------------

We've concentrated on P, NP, and PSPACE because they are all based on
polynomial resource bounds, and our interest was in efficient computation.
More importantly, a vast majority of "real-life" problems that arise
naturally from various application domains belong to NP.

We have seen how to prove problems are NP-complete (and examples of
PSPACE-complete problems) and why this is evidence that these problems have
no efficient solution.  But are there problems that can be proved to have
no efficient solution unconditionally?

Definitions:
    EXP = U TIME(2^{n^k})
        = { languages decided by TMs in time O(2^{n^k}) }
    NEXP = U NTIME(2^{n^k})
         = { languages decided by NTMs in time O(2^{n^k}) }
    EXPSPACE = U SPACE(2^{n^k})
             = { languages decided by TMs in space O(2^{n^k}) }

By Savitch's Theorem, NEXPSPACE = EXPSPACE.

Known: L (_ NL (_ P (_ NP (_ PSPACE (_ EXP (_ NEXP (_ EXPSPACE

Unknown: L ?= NL, NL ?= P, P ?= NP, NP ?= PSPACE, PSPACE ?= EXP,
         EXP ?= NEXP, NEXP ?= EXPSPACE
In other words, we don't know how to prove that nondeterminism makes a
difference, and we don't know how to prove that space is more powerful than
time.

Known: NL != PSPACE, P != EXP, NP != NEXP, PSPACE != EXPSPACE
In other words, we can prove that exponential gaps make a difference
(when measuring the same resource bound).

How do we know these results?  Because of so-called "hierarchy theorems":
for most functions f_1, f_2 such that f_1 in o(f_2) (i.e., lim_{n -> oo}
f_1(n)/f_2(n) = 0), TIME(f_1) is a proper subset of TIME(f_2) and
SPACE(f_1) is a proper subset of SPACE(f_2).
Idea: construct a decider D that runs in time (or space) O(f_2(n)) and
behaves as follows: on input <M>, D simulates M on input <M> for at most
f_1(n) steps (or using at most f_1(n) space), and accepts if M rejects,
rejects if M accepts (or exceeds the time or space bound).  Then, L(D) in
TIME(f_2) (or SPACE(f_2)) but L(D) must be different from every language in
TIME(f_1) (or SPACE(f_1)), by construction.  (See section 9.1 in textbook
for technical details required to make this work.)

Problems that are complete for EXP are known to be not in P, e.g.,
"generalized chess", "generalized checkers".  Similarly, problems that are
complete for EXPSPACE require more than polynomial space and are highly
intractable, e.g., "inequivalence of regular expressions with squaring",
"equivalence of regular expressions with exponentiation".

But it doesn't stop there!  We can define k-EXP, k-NEXP, k-EXPSPACE
(deterministic time, nondeterministic time, space) 2^(2^...(2^n^t)...),
where there are k exponentiations (so EXP = 1-EXP), and show there are
languages in each class that are not contained in smaller classes.
Even more, ELEMENTARY = 1-EXP U 2-EXP U ... = 1-EXPSPACE U 2-EXPSPACE U ...
(since k-EXPSPACE is a subset of (k+1)-EXP).  Problems complete for
ELEMENTARY are decidable, but with such astronomical time or space bounds
that they are completely intractable.  Yet, "inequivalence of regular
expressions with union, concatenation, and negation" requires running time
2^(2^(...2^(2^n)...)) where there are at least log n many exponentiations,
so it's outside even ELEMENTARY -- yet still decidable!