Complexity of arithmetic operations

If a and b are, for example, floats, then the complexity of a*b or a+b is $\mathcal{O}(1)$. That's because, for numbers that have a set number of digits, operations like addition and multiplication are performed by the CPU (the Central Processing Unit.)

To simplify a little bit, if the CPU's clock speed is 2Ghz, that means it can perform 2 billion elementary operations like addition or multiplication per second.

However, for ints, the story is more complicated, since they do not have a set number of digits, and can be arbitrarily large. How long does it take to perform addition?

An efficient way to perform addition is long addition -- adding the numbers digit-by-digit.

    123987
   +654654 
   =======
    778641

Roughly speaking, a+b requires as many additions as there are digits in $\max(a, b)$.

There are approx $\log_{10} a$ decimal digits in the number $a$. So the complexity of addition is $\mathcal{O}(\max(\log a, \log b)$.

How about multiplication? Here is a naive approach:

In [1]:
def mult(a, b):
    prod = 0
    for i in range(a):
        prod += b
    return prod

We repeat the iteration $a$ times, and each time we perform a number of operations that's proportional to at most $\log ab = \log a + \log b$. So the complexity is $\mathcal{O}(a\log ab)$.

Can we do better? Yes. That requires the use of long multiplication. To remind you, long multiplication goes like this:

  123
  521
  ====
   123
  246        
 615
 ======
 64083

To multiply two three-digit numbers, we needed to perform $3\times 3$ multiplications of single digits (and also about the same number of additions.) In general, we'd need about $\log a \times \log b$ multiplications and additions.

So the complexity of long multiplication is $\mathcal{O}(\log a \times \log b)$. If the the number of digits in $a$ and $b$ is $n$, we can say the complexity is $\mathcal{O}(n^2)$.

Is it the best we can do? In fact, no. Algorithms that run in, e.g., $\mathcal{O}(n^{1.59})$ time exist as well.

Here is a larger example, with multiplication of two 6-digit numbers:

        d1 d2 d3 d4 d5 d6
                    D1 D2 D3 D4 D5 D6
                    =================
                    p6 p5 p4 p3 p2 p1
              p12 p11 p10 p9 p8 p7
         .....................                  <-----   36 products
      ...................
   p36 p35 p34 p33 p32 p31 
   ===================================
   s1 s2 s3 s4 s5 .......         sk           <- to compute all of s1, s2, s3, ..., sk, we add in each p_i once, and we have approx 36 p_i's