We showed a bunch of languages were regular...
However, from lecture 1, we know that there are some problems that computers can’t solve...
... so what do non-regular languages look like?
What are some limitations for DFAs and NFAs?
Regular languages KEY intuition
DFAs has a finite number of states.
States correspond to memory.
Thus, DFAs can compute languages that only need a finite amount of memory (and read the input once left to right).
In particular, a DFA has a fixed amount of memory, no matter how large the input is.
Example
\(\mathbf{Even}\) is regular because no matter how large the input is, I only need to store one bit corresponding to whether or not the input has an even number of \(1\)s so far.
Infinite Memory Required
What are some things you can’t do with a fixed amount of memory?
Solution
Even simple things like storing the length of the input or the number of 1s - we don’t know in advance how long our input string can be!
Example
Here’s an example of a language that can’t be computed using finite memory.
\[
\{a^nb^n: n \in \mathbb{N}\}
\]
Why?
I don’t know ahead of time how many \(a\)s there are, and I need to keep track of them to see how many \(b\)s I should expect.
Proving not regular
Intuitively, \[
X = \{a^nb^n: n \in \mathbb{N}\}
\]
requires infinite memory so is not regular.
To show \(X\) is not regular, we need to show that there does not exist a DFA \(M\) such that \(L(M) = X\).
\(X = \{a^nb^n: n \in \mathbb{N}\}\) is not regular
Solution
By contradiction, suppose there was a DFA \(M\) such that \(L(M) = X\).
Claim: Suppose \(m, n \in \mathbb{N}\) such that \(m \neq n\), then \(M\) run on \(a^m\) and \(a^n\) end up in different states.
Proof of claim. Let \(q_m\), \(q_n\) be the states reached after reading \(a^m\) and \(a^n\) respectively. By contradiction, suppose \(q_m = q_n\). Suppose from this state, we then read \(b^m\), let \(q'\) be the final state. Since \(a^mb^m \in X\), \(q'\) should be accepting. However, since \(a^nb^m \notin X\), \(q'\) should be rejecting, we have reached a contradiction since \(q'\) cannot be both.
By the claim, the DFA must reach a unique state for each \(a, aa, aaa,...\). Thus, \(M\) must have infinitely many states, which is a contradiction since \(M\) is supposed to be a DFA.
Key Insights
Same state \(\implies\) same fate. If two strings \(x, y\) led the DFA to the same state. No matter what string \(w\) was read after, either \(xw\) and \(yw\) both get accepted or \(yw\) both get rejected.
The language \(\{a^nb^n: n \in \mathbb{N}\}\) had infinitely many strings that do NOT share the same fate (and hence must have distinct states).
Myhill-Nerode Theorem (corollary)
Let \(A\) be a language over \(\Sigma\). Suppose there exists a set \(S \subseteq\Sigma^*\) with the following properties
(Infinite). \(S\) is infinite
(Pairwise distinguishable). \(\forall x, y \in S\), with \(x \neq y\). \(x\), and \(y\) are distinguishable relative to \(A\).
Then \(A\) is not regular.
Proof
Solution
Let \(A\) be language, and suppose \(S \subseteq\Sigma^*\) is infinite and pairwise distinguishable relative to \(A\). WTS \(A\) is not regular.
By contradiction, suppose \(A\) was regular, then there exists some DFA \(M\) such that \(L(M) = A\). Since \(M\) is a DFA, it has some finite set of states \(Q\).
Let \(g: S \to Q\), be a function mapping strings \(x \in S\) to the state the DFA reaches after reading \(x\) from the start state.
Since \(S\) is infinite, and \(Q\) is finite, \(g\) is not injective. Therefore, there exist two strings \(x, y \in S\) such that \(g(x) = g(y)\). That is \(x\) and \(y\) reach the same state. Since \(x\) and \(y\) are in \(S\), they are distinguishable; however, by the lemma, they are indistinguishable, which is a contradiction.
Using The Myhill-Nerode Theorem
To show a \(A\) is not regular, it suffices (by the Myhill-Nerode Theorem) to find a set of strings, \(S\), such that \(S\) is infinite and pairwise distinguishable relative to \(A\).
Proof: \(X = \{a^nb^n: n \in \mathbb{N}\}\) is not regular
Solution
Consider the set \(S = \{a^n: n \in \mathbb{N}\}\) and note that \(S\) is infinite since it has one element for every natural number.
Let \(x, y \in S\) with \(x \neq y\). Then \(x = a^i\), \(y = a^j\) for some \(i \neq j\). We’ll show that \(x \not \sim y\), in particular, \(w = b^i\) is such that \(xw = a^ib^i \in X\), but \(yw = a^jb^i \notin X\). Thus, \(S\) is an infinite set of pairwise distinguishable strings relative to \(X\), so \(X\) is not regular.
A note about the pumping lemma
The pumping lemma is an alternate way to show that a language is not regular.
I used to cover both the pumping lemma and the Myhill-Nerode theorem in class, but I found that students found the Myhill-Nerode theorem easier to understand and use.
You do not need to know the pumping lemma for this course, but if you’re interested, there is a chapter in Sipser that covers it.
Final regular expression: \(1^+0 | 0(\epsilon | 1)\)