=========================================================================== CSC 363 Lecture Summary for Week 1 Fall 2009 =========================================================================== Administrivia - Material is not that difficult, but very abstract: you must keep up in order to understand. In particular, do the readings and take the time to work on the homework carefully. Also, please let me know when things don't make sense (don't be shy) so that I can provide additional explanations or examples. - Focus on learning rather than evaluation, for numerous small exercises. Evaluation mainly from assignments, tests, exam. - Please don't hesitate to get in touch with me with any concerns or suggestions you may have about any aspect of the course: if there are aspects you particularly like, I want to know so that I can keep them the same; if there are aspects you particularly dislike, I want to know so that I can make changes (or explain my reasons for having things this way) -- don't wait until the official course evaluations at the end of term, when it's too late to make a difference now. - LAPTOPS: Because of a bad experience in the past, I'd like to ask you to please not show up just for the sake of showing up and spend your time working/chatting/playing (rather than paying attention and taking notes) -- it's a distraction for other students and kind of insulting for me. Also, multi-tasking is a myth! Studies show that trying to focus on more than one thing at once actually decreases productivity -- it's better to do one thing at a time, and do it well. I don't take attendance, and my lecture notes are posted online: I would think it more considerate for someone to miss a lecture than to show up with no intention of participating. If you absolutely must show up and work on something else, at least please be considerate and sit at the back of the room to avoid distracting others. Prerequisites: CSC 236H/238H/240H is necessary -- need good background in proving algorithm correctness, analysis of algorithm complexity, and formal language theory (regular languages, context-free languages). Similarly, CSC263H and CSC373H are an excellent experience to have under your belt before you take this course, though not strictly required. --------------------------- Computational Computability --------------------------- Outline (topics and textbook sections): 1. Turing machines: definitions, examples (3.1) 2. Variants, the Church-Turing thesis (3.2, 3.3) 3. the Halting problem (4.1, 4.2) 4. Decidability and recognizability, examples (4.2, 5.1) 5. Reducibility, examples (5.1, 5.2) 6. Mapping reducibility, examples (5.3) --------------- Turing machines --------------- Motivation: - Goal: define "computation" as abstractly and generally as possible. - Many possible formalizations: start with one, study it, then compare with others. Informal idea: similar to FSA but with no limitation on access to input: - one-way infinite "tape" divided into "squares" (or "cells") (each square holds one symbol); - read-write "head" positioned on one square at a time; - "control" can be in one of a fixed number of states; - initially, tape contains input (one symbol per square, starting on leftmost square) and blanks, and head is on leftmost input symbol; - current state and symbol read determine next state, symbol written, and movement of head (one square left or right). Differences between FSA and Turing machines: - TM can both read and write symbols. - Infinite tape. - Head can move left or right (convention: moving left on leftmost square leaves head where it is). - Special "accept" and "reject" states that stop computation immediately. Example: M_1 that accepts exactly strings of the form w#w for w (- {0,1}* Read first symbol and cross it off (replace with new symbol 'x'), move right until #, keep moving right until first non-x symbol to verify same as first symbol seen earlier (remembered through states), cross it off and go back to leftmost non-x symbol to repeat. If more than one # or different symbols or more symbols on one side than the other, reject; otherwise, accept. Formal definition: - A TM is a 7-tuple (Q,\Sigma,\Gamma,\delta,q_0,q_accept,q_reject), where . Q: fixed, finite, non-empty set of "states" . q_0 (- Q: "start state" (or "initial state") . q_accept (- Q: "accepting state" . q_reject (- Q: "rejecting state" . (q_reject != q_accept) . \Sigma: fixed, finite, non-empty set of symbols ("input alphabet") . \Gamma: fixed, finite, non-empty set of symbols ("tape alphabet"), . \Sigma (_ \Gamma, "blank symbol" _ (- \Gamma, _ !(- \Sigma . \delta : Q-{q_accept,q_reject} x \Gamma -> Q x \Gamma x {L,R} ("transition function") - A "configuration" of a TM represents the current state, tape content, and head position as a string "u q v", where q is current state, uv is current tape content (from the leftmost square up to the infinitely many blanks, not indicated), and head is on leftmost symbol of v. Note: because tape is infinite to the right, configuration "u q v" is equivalent to "u q v_", "u q v__", "u q v___", etc. - On input w, TM M computes as follows: . start from initial (or "start") configuration C_0 = "q_0 w" [ALTERNATE CONVENTION: C_0 = "_ q_0 w", to simplify computation]; . go from configuration to configuration following the transition function (i.e., C_0 yields C_1 yields C_2 yields ...); more precisely, for all states q_i (- Q-{q_accept,q_reject}, q_j (- Q, symbols a, b, c (- \Gamma, strings u, v (- \Gamma*, . if \delta(q_i,b) = (q_j,c,R), then configuration "u q_i bv" yields "uc q_j v" in one step of computation; . if \delta(q_i,b) = (q_j,c,L), then configuration "ua q_i bv" yields "u q_j acv" in one step of computation and configuration "q_i bv" yields "q_j cv" (head cannot move "off" left end); . three possible outcomes: - TM "accepts" w if it reaches some accepting configuration C_m = "u q_accept v" (at which point computation stops); - TM "rejects" w if it reaches some rejecting configuration C_m = "u q_reject v" (at which point computation stops); - TM "loops on" w if it never reaches an accepting or rejecting configuration -- this is NOT signalled by a specific loop of configurations, just the absence of accepting or rejecting. Definitions: - Recall: "language" is any subset of \Sigma* (set of all strings over alphabet \Sigma). - The language "recognized" by a TM M is L(M) = { w (- \Sigma* : M accepts input w }. - L is "Turing-recognizable" (recognizable/semi-decidable/recursively enumerable) if there is some TM M such that L = L(M), i.e., if there is some TM M such that for all strings w (- \Sigma*, M accepts w when w (- L and M rejects or loops on w when w !(- L. - L is "Turing-decidable" (decidable/recursive) if there is some TM M that halts on every input such that L = L(M), i.e., for all strings w (- \Sigma*, M accepts w when w (- L and M rejects w when w !(-L. Such a TM is called a "decider". Example 3.7 on pp.143-144: TM to recognize language { 0^{2^n} : n >= 0 }, i.e., TM that accepts all strings that contain a power of 2 many 0s, but no others. High-level description (one-paragraph algorithm): repeatedly cross off every second 0 (cuts down number of 0s in half), until only one 0 remains, by going back-and-forth over input; if at any point, number of 0s is odd and > 1, reject; otherwise, accept when number of 0s reaches exactly 1. Implementation-level description (algorithm in stages, with details of head movement and tape contents but no mention of states): On input w, 1. Replace first symbol of w with _, move right, enter stage 2 in "crossing off" mode (leftmost square represents a 0). 2. Move right, alternating between "skipping" and "crossing off" each 0 encountered, and ignoring Xs, until _ at end of input. - If last action was "skipping", accept if there was exactly one 0 skipped; reject otherwise (number of 0s is odd and > 1). - Otherwise, go to stage 3. 3. Move head left until we reach blank on leftmost square, move one square right and enter stage 2 in "crossing off" mode. [NOTE: What follows was not covered in detail during lecture. Read it carefully and don't hesitate to ask questions if anything doesn't make sense.] Formal-level description (full transition table/diagram): Q = {q_1,q_2,q_3,q_4,q_5,ACC,REJ} NOTE: Just like for FSA, can't know ahead of time how many states are needed; we find out as we design the TM. \Sigma = {0}; \Gamma = {0,x,_} start state = q_1; accepting state = ACC; rejecting state = REJ \delta described by table below (easier than picture in ASCII!) -- find current state down left side and current symbol across top, entry at that row/column gives next state, symbol, head move. NOTES: comments are for human readability and not technically part of formal description; transitions surrounded by square brackets [] indicate situation that cannot happen by design of the TM (called "superfluous" transitions) -- details of these transitions are meaningless; nevertheless, they must be specified in formal description, so square brackets serve same purpose as comments (for human readability). For instance, it is impossible to read symbol 'x' while in state q_1 because by convention, TM tape can only contain symbols from input alphabet initially, and x is not in \Sigma (also, q_1 only used for initial state: no transition ever goes back to q_1). 0 x _ replace first symbol of input with _ and move right q_1 q_2,_,R [q_1,x,R] REJ,_,R cross off next 0 (only leftmost 0 skipped; accept if no other) q_2 q_3,x,R q_2,x,R ACC,_,R skip next 0 (at least one skipped, one crossed) q_3 q_4,0,R q_3,x,R q_5,_,L cross off next 0 (at least one crossed, one skipped) q_4 q_3,x,R q_4,x,R REJ,_,R "rewind" head to the left, start over q_5 q_5,0,L q_5,x,L q_2,_,R Trace on input 0000, using notation introduced earlier (configurations) q_1 0000_ _ q_2 000_ _x q_3 00_ _x0 q_4 0_ _x0x q_3 _ _x0 q_5 x_ _x q_5 0x_ _ q_5 x0x_ q_5 _x0x_ _ q_2 x0x_ _x q_2 0x_ _xx q_3 x_ _xxx q_3 _ _xx q_5 x_ _x q_5 xx_ _ q_5 xxx_ q_5 _xxx_ _ q_2 xxx_ _x q_2 xx_ _xx q_2 x_ _xxx q_2 _ _xxx_ ACC