Notes from Monday, February 12, 2006

Introduction to the analysis of algorithms

---------------------------------------------------------------------
Consider the following formula for computing an approximation to e:

       1     1     1     1     1
  e = --- + --- + --- + --- + --- + ...
       0!    1!    2!    3!    4!

---------------------------------------------------------------------
Algorithm factorial(K):

input K
set PRODUCT to 1
set N to 1
while N <= K do
   set PRODUCT to PRODUCT*N
   set N to N+1
(end of loop)
output PRODUCT

# of loop cycles = K
# of multiplications = K
# of additions = K
# of comparisons = K+1


Algorithm eApprox1(N):

input N (the number of terms to add up)
set SUM to 0
set D to 0
while D < N do
   set SUM to SUM + 1/factorial(D)
   set D to D+1
(end of loop)
output SUM

# of loop cycles (D goes from 0 to N-1) = N
# of multiplications = 0 + 1 + 2 + ... + N-1 = ???

+--+--+--+--+
|  |xx|xx|xx|
+--+--+--+--+
|  |  |xx|xx|
+--+--+--+--+ N units
|  |  |  |xx|
+--+--+--+--+
|  |  |  |  |
+--+--+--+--+
   N units

  number of blank squares = 1 + 2 + ... + N-1 + N
+    number of xx squares = 1 + 2 + ... + N-1
-----------------------------------------------------
= total number of squares = 2*(1 + 2 + ... + N-1) + N
                          = N^2

Thus 1 + 2 + ... + N-1 = (N^2 - N)/2 = 1/2 N^2 - 1/2 N

So eApprox1(N) requires 1/2 N^2 - 1/2 N multiplications in all

Other possible measures of running time:

  # of multiplications = 1/2 N^2 - 1/2 N

  # of additions = [0+1+2+...+(N-1)] + 2N
                 = 1/2 N^2 + 3/2 N

  # of divisions = N

  # of arithmetic operations = multiplications + additions + divisions
                             = 1/2 N^2 - 1/2 N + 1/2 N^2 + 3/2 N + N
                             = N^2 + 2N

  # of comparisons = N+1 + [(0+1)+(1+1)+(2+1)+...+(N-1)+1]
                   = 1/2 N^2 + 3/2 N + 1

  # of everything = arithmetic ops + comparisons
                  = N^2 + 2N + 1/2 N^2 + 3/2 N + 1
                  = 3/2 N^2 + 7/2 N + 1

No matter how you slice it, you end up with an N^2 term

We say that the running time of eApprox1 is "order N squared" or
"quadratic", written O(N^2) or Theta(N^2)

---------------------------------------------------------------------
Now consider an alternative algorithm:

Algorithm eApprox2(N):

input N (the number of terms to add up)
set SUM to 0
set DENOM to 1
set i to 1
while i <= N do
   set SUM to SUM + 1/DENOM
   set DENOM to DENOM*i
   set i to i+1
(end of loop)
output SUM

# loop cycles = N
# multiplications = N
# additions = 2N
# divisions = N
# arithmetic operations = 4N
# comparisons = N+1
# everything = arithmetic ops + comparisons = 5N+1

Running time of eApprox2 is "order N" or "linear",
written O(N) or Theta(N)

---------------------------------------------------------------------