How I Go From 70 Lines Of Code To Only 26 Using The NumPy Library
Leo Gau
Posted on April 3, 2021
Context
In my previous post I implemented a nerual network to understand handwritten digits using matrix math functions I implemented myself.
Luckily, I don't have to be doing this going forward in the future. The NumPy library will do this all for me. Numpy is the one of the foundational libraries in the Python scientific computing ecosystem. It provides a high-performance, multidimentional array object which makes it fast and easy to work with matrices.
In this post, I'll show you how I use NumPy to replace the hand written math functions I wrote.
The Original Code Using Only Python
Below is the original code with my owm matrix multiplication functions.
def flatten_image(image):
return list(itertools.chain.from_iterable(image))
def weighted_sum(a, b):
assert(len(a) == len(b))
output = 0
for i in range(len(a)):
output += (a[i] * b[i])
return output
def vector_matrix_multiplication(a, b):
output = [0 for i in range(10)]
for i in range(len(output)):
assert(len(a) == len(b[i]))
output[i] = weighted_sum(a, b[i])
return output
def zeros_matrix(rows, cols):
output = []
for r in range(rows):
output.append([0 for col in range(cols)])
return output
def outer_product(a, b):
output = zeros_matrix(len(a), len(b))
for i in range(len(a)):
for j in range(len(b)):
output[i][j] = a[i] * b[j]
return output
class NeuralNet:
def __init__(self):
self.weights = [
[0.0000 for i in range(784)],
[0.0001 for i in range(784)],
[0.0002 for i in range(784)],
[0.0003 for i in range(784)],
[0.0004 for i in range(784)],
[0.0005 for i in range(784)],
[0.0006 for i in range(784)],
[0.0007 for i in range(784)],
[0.0008 for i in range(784)],
[0.0009 for i in range(784)]
]
self.alpha = 0.0000001
def predict(self, input):
return vector_matrix_multiplication(input, self.weights)
def train(self, input, labels, epochs):
for i in range(epochs):
for j in range(len(input)):
pred = self.predict(input[j])
label = labels[j]
goal = [0 for k in range(10)]
goal[label] = 1
error = [0 for k in range(10)]
delta = [0 for k in range(10)]
for a in range(len(goal)):
delta[a] = pred[a] - goal[a]
error[a] = delta[a] ** 2
weight_deltas = outer_product(delta, input[j])
for x in range(len(self.weights)):
for y in range(len(self.weights[0])):
self.weights[x][y] -= (self.alpha * weight_deltas[x][y])
Implement The Helper Functions With NumPy
To help me better understand how NumPy slots into this code, I'm going to keep my helper functions but implement them using NumPy. For example, I still have a weighted sum function but instead of hand calculating the weighted sum, I use the NumPy dot function.
def flatten_image(image):
return image.reshape(1, 28*28)
def weighted_sum(a, b):
return a.dot(b)
def vector_matrix_multiplication(a, b):
return np.matmul(input, weights.T)
def zeros_matrix(rows, cols):
return np.zeros((rows, cols))
def outer_product(a, b):
return np.outer(a, b)
class NeuralNet:
def __init__(self):
self.weights = np.random.random((10, 28 * 28)) * 0.0001
self.alpha = 0.0000001
def predict(self, input):
return vector_matrix_multiplication(input, self.weights)
def train(self, input, labels, epochs):
for i in range(epochs):
for j in range(len(input)):
pred = self.predict(input[j])
label = labels[j]
goal = np.zeros(10)
goal[label] = 1
delta = pred - goal
error = delta ** 2
weight_deltas = outer_product(delta, input[j])
self.weights -= (self.alpha * weight_deltas)
Use The NumPy Functions Inline
Already you can see the code is a lot cleaner.
If we remove those helper functions and do everything inline, the code shrinks even more. The original implmentation is 70 lines long and this one is only 26 lines.
The NumPy library is doing a lot of work for me.
def flatten_image(image):
return image.reshape(1, 28*28)
class NeuralNet:
def __init__(self):
self.weights = np.random.random((10, 28 * 28)) * 0.0001
self.alpha = 0.0000001
def predict(self, input):
return np.matmul(input, self.weights.T)
def train(self, input, labels, epochs):
for i in range(epochs):
for j in range(len(input)):
pred = self.predict(input[j])
label = labels[j]
goal = np.zeros(10)
goal[label] = 1
delta = pred - goal
error = delta ** 2
weight_deltas = np.outer(delta, input[j])
self.weights -= (self.alpha * weight_deltas)
import itertools
import numpy as np
from keras.datasets import mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
images = x_train
labels = y_train
prepared_images = [flatten_image(image) for image in images]
prepared_labels = np.array(labels)
nn = NeuralNet()
nn.train(prepared_images, prepared_labels, 5)
test_set = x_test
test_labels = y_test
num_correct = 0
for i in range(len(test_set)):
prediction = nn.predict(flatten_image(test_set[i]))
correct = test_labels[i]
if np.argmax(prediction) == int(correct):
num_correct += 1
print(str(num_correct/len(test_set) * 100) + "%")
76.05%
So What's Next?
The code is now so efficient I can train on the full dataset and the accuracy gets a little bump.
In the next post we'll see if adding a layer helps improve this accuracy even more.
Posted on April 3, 2021
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
