Notes from Thursday, February 15, 2006

====================================================================
BINARY SEARCH

- searching for a value in a sorted list (e.g. a telephone directory)
- search process repeatedly halves the size of the problem
- number of steps in the worse case is the base 2 logarithm of the
  problem size (the number of elements in the list)

====================================================================
FAST EXPONENTIATION

Algorithm Power(BASE, N):

set PRODUCT to 1
set COUNT to 1
while COUNT <= N do
   set PRODUCT to PRODUCT*BASE
   set COUNT to COUNT+1
(end of loop)
output PRODUCT

Computing Power(2, 1000000) would require 1000000 steps

Algorithm Fastpower(BASE, N):

set PRODUCT to 1
while N > 0 do
   if N is even then
      set BASE to BASE*BASE
      set N to N/2
   else
      set PRODUCT to PRODUCT*BASE
      set N to N-1
(end of loop)
output PRODUCT

Best case example: N=32 takes 6 steps

N=32 -> N=16 -> N=8 -> N=4 -> N=2 -> N=1 -> N=0

...about log(n) steps in the best case (log(n)+1 exactly)

Worst case example: N=31 takes 9 steps

N=31 -> N=30 -> N=15 -> N=14 -> N=7 -> N=6 -> N=3 -> N=2 -> N=1 -> N=0

...about 2 log(n) steps in the worst case (2*floor[log(n)]+1 exactly)

O(log n) complexity

Computing Fastpower(2, 1000000) would take on the order of
log(1000000) steps, which is about 20.  Much faster than Power!

====================================================================
PRIME TESTING - How to determine if N is prime?

Here is the algorithm we discussed last week for testing primality:

Algorithm PrimeTester(N):

if N < 2 then print "not prime"
else if N = 2 then print "is prime"
else
   set D to 3
   set PRIME to True
   while D <= sqrt(N) and PRIME is True do    ("sqrt" = "square root")
      if N/D has remainder 0 then
         print "not prime"
         set PRIME to False
      else
         set D to D+2
   (end of loop)
   if PRIME is still True then print "is prime"

Worst case example: N is prime

D goes from 3, 5, 7, 9, ... up to sqrt(N)

...about 1/2 sqrt(N) loop cycles in all

O(sqrt n) complexity

====================================================================
FAST EXPONENTIATION, MODULO M

 1000
2     mod 10 = ???

Exponentiation modulo some number M is just like regular exponentiation,
except we always keep the values within the range 0...M-1 by dividing
each intermediate result by M and keeping just the remainder.

regular            modulo 10
 2 * 2 = 4         2 * 2 mod 10 =  4 mod 10 = 4  (4/10 has remainder 4)
 4 * 2 = 8         4 * 2 mod 10 =  8 mod 10 = 8  (8/10 has remainder 8)
 8 * 2 = 16        8 * 2 mod 10 = 16 mod 10 = 6  (16/10 has remainder 6)
16 * 2 = 32        6 * 2 mod 10 = 12 mod 10 = 2  (12/10 has remainder 2)
32 * 2 = 64        2 * 2 mod 10 =  4 mod 10 = 4  (4/10 has remainder 4)
64 * 2 = 128       4 * 2 mod 10 =  8 mod 10 = 8  (8/10 has remainder 8)
.                  .
.                  .
.                  .

Algorithm Powermod(BASE, N, MOD):

set PRODUCT to 1
while N > 0 do
   if N is even then
      set BASE to BASE*BASE
      set BASE to remainder of BASE/MOD         <-- added this line
      set N to N/2
   else
      set PRODUCT to PRODUCT*BASE
      set PRODUCT to remainder of PRODUCT/MOD   <-- added this line
      set N to N-1
(end of loop)
output PRODUCT

Like Fastpower, Powermod takes about 2 log(n) steps in the worst case

O(log n) complexity

====================================================================
PROBABILISTIC ALGORITHMS - the Fermat prime test

Pierre de Fermat (17th century):
If N is a prime number, then the relation
   a^N mod N = a
holds for all numbers from 1 to N-1.  On the other hand, if N is
not prime, then this relation will USUALLY be false for almost all
numbers from 1 to N-1.

Idea: pick a number from 1 to N-1 at random and see if the relation
holds.  If it doesn't, we know N isn't prime.  Otherwise, try a few
more spot checks before concluding that N is prime.

Algorithm Fermat(N):

if N < 2 then print "not prime"
else
   set TRIES to 0
   set PRIME to True
   while TRIES < 10 and PRIME is True do
      choose a random number A in the range 1 <= A <= N-1
      if Powermod(A, N, N) does not equal A then
         print "not prime"
         set PRIME to False
      else
         set TRIES to TRIES+1
   (end of loop)
   if PRIME is still True then print "is prime"

Worst case running time is 10 * (2 log N)

O(log N) complexity

This is MUCH more efficient than O(sqrt N) for large values of N

Carmichael numbers fool the Fermat test: all of the values from 1 to N-1
satisfy the "a^N mod N = a" relation, but N is still not prime!

Carmichael numbers are very rare: only 255 of them below 100,000,000:
   561, 1105, 1729, 2465, 2821, 6601, ...