Memoization in a Nutshell
Pan Chasinga
Posted on July 14, 2018
If anyone had ever set eyes on Cracking the Coding Interviews (or any other books on algorithms), you might at least know what memoization is and what it's good for. Consider this a more comprehensive, layman, JavaScript version of the excerpt.
There's no problem that can explain (bad) recursion simpler than finding the n
fibonacci number, so here goes:
function fib(n) {
if (n == 0 || n == 1) return n;
return fib(n - 1) + fib(n - 2);
}
Oh no, the syntax highlighting isn't working for you? No worries, that was just me. IMO, highlighting is quite distracting for reading code especially this small. It is helpful in writing, but you want to focus on the logical steps and make sure you get them while reading.
It's pretty straightforward. A recursive function with one or more base cases (in this case, when n == 0
and n == 1
) and the recursive one. If you have no clue what how recursion works at all, I recommend a soft consultation of this post: Recursion Made Simple. (And don't forget to clap and follow me. It takes time and effort to write!)
The problem with the said fib
function is it runs at exponential time O(2^n)
. That's like (almost) the worst runtime you could get into. How bad exactly? If you called fib(50)
to compute the 50th fibonacci number and it took a minute to return the result, then calling fib(100)
will take you roughly 1,125,899,906,000,000 minutes (rounded down to the millionth), which is just around 2 billion years (Fun fact: By then our earth and half of the solar system should have been eaten by the growing sun long ago).
To be fair, this problem is an intentional bad candidate for recursion. This recursive part:
fib(n - 1) + fib(n - 2);
means that for every N
th node of the function call, two more branch out. To make this worse, for each N
th call, there are works repeated. Below is a painstakingly created ASCII tree diagram of what really happens:
fib(5)
/ \
fib(4) fib(3)
/ \ / \
fib(3) fib(2) fib(2) fib(1)
/ \ / \ / \
/ \ fib(1) fib(0) fib(1) fib(0)
fib(2) fib(1)
/ \
fib(1) fib(0)
As you can see, the work done from fib(3)
down could be done once instead of repeated the way it is. When N = 5
, you could see that fib(N - 2)
is being computed twice, fib(N - 3)
three times. If this goes on long enough, says N
is a high number like 100, you can be sure that
For every
fib(N)
,fib(N - (N - p))
will be repeatedp
times wherep ∈ {x | 2 < x < N}
(p
is element ofrange (2..N-1)
). Talking about baggages!
Memoization = Memorizing the past
As melodramatic as it may sound, that sums up the definition of this technique. Imagine your code is equipped with AI capability to not repeat the amount of work we mentioned. Still, the AI needs to have a way of memorizing what has been done. In this case, the fictitious AI isn't going to be of much help. The smartest thing it could do is to realize that the fib
operation is a suicidal mission and switch to memoization mode from the get.
And what's the best way our brain remember and recall a memory quickly? By associating it with something else. That is exactly how associate arrays (hash map, hash table, dictionary) and arrays work!
In our fibonacci case, we could use either data structure, but array is simpler since the keys are already integers.
The concept is to have the fib
function "carries along" an array filled with past memories so at any moment in its mesmerizing recursive life, it can recall that a work it's about to do had actually been done already and should just be lazy about it. Here is how it's done:
function fib(n, brain = []) {
if (n == 0 || n == 1) return n;
// If brain has no memory of the work for fib(n)
if (brain[n] == 0) {
// compute and then memorize it
brain[n] = fib(n - 1, brain) + fib(n - 2, brain);
}
// just returns what's already memorized
return brain[n];
}
Now whenever fib
gets called, it carries along a brain
with memories of the past work, just so that it could avoid doing work more than once. This will of course come with having to sacrifice a linear amount of space for brain
array, but now you gets to finish computing fib(10000)
in fractions of a millisecond instead of waiting for the universe to die out twice over.
p.s. I'll leave it to you to figure out the new runtime of our "mindful" fib
.
ciaos
Posted on July 14, 2018
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.