Learn Python by building investment AI for fintech - Lesson4: Recurrent Neural Network (RNN)

This article was originally published at:
https://www.blog.duomly.com/python-course-with-building-a-fintech-investment-ai-lesson-4-recurrent-neural-network-rnn

Intro

In today's episode of the Python course with AI, we will learn how to make predictions with AI, and we will predict the prices.

Last episodes you can find here:

Lesson 3 - How to create and train AI:

Python course with building a fintech investment AI – Lesson 3: How to build and train AI

Lesson 2 - How to find financial data and how to use Pandas:

Python course with building a fintech investment AI – Lesson 2: Pandas and getting financial data

Lesson 1 - Prepare the AI project:

Python course with building a fintech investment AI – Lesson 1: Start the project

In the fourth lesson of the AI course, we will use the AI that we created in the previous episode.

We will predict some first stock prices and will compare with the real ones, to have an idea if our AI is smart.

Let's start!

If you prefer video, here is the youtube version:

Prepare data for prediction

As the first step, we should create a function that will prepare data for the prediction.

Create a function named "preparePredictData" in the file ai.py, below function "trainModel".

Next, pass "stockPricesLastMonth", and "stockPricesTest" as params.

def preparePredictData(stockPricesLastMonth, stockPricesTest):

Take all data

As the second step of preparing data, we should concatenate all the data that we have into the one data frame.

We can use the pandas "concat" method for that.

Next, we need to create the variable named "predictInputs".

To this variable, we should assign all the items from the position that we will get by calculation, up to the end of the data frame.

The calculation is the length of the total data, minus length of the lastMonth data, minus 120(6 months).

  dataset_total = pd.concat((stockPricesTest['Open'], stockPricesLastMonth['Open']), axis = 0)
  predictInputs = dataset_total[len(dataset_total) - len(stockPricesLastMonth) - 120:].values

Reshape the data

In the third step of the data preparation, we should reshape our data by using "reshape" method.

That will give us the formatted list of the "predictInputs", and our data will be ready to be scaled later.

predictInputs = predictInputs.reshape(-1,1)

Scale data to fit this one from training and prepare testPredicts

The fourth step should be to take 20 items from the "predictInputs", and append them into the array named "testPredicts".

predictInputs = scaler.transform(predictInputs)
testPredicts = []

for i in range(120, 140):
    testPredicts.append(predictInputs[i-120:i, 0])

Make 3d array and reshape

As the last step of the data preparation, we need to do 3d array from those items and reshape them.

Next, we need to return "testPredicts" variable.

testPredicts = np.array(testPredicts)
testPredicts = np.reshape(testPredicts, (testPredicts.shape[0], testPredicts.shape[1], 1))

return testPredicts

Make prediction

Super, we are done with data preparation now!

We can create the logic that will predict the prices for us.

After the prediction, we cannot forget to reinverse the data to the values that will be human-readable.

def getPrediction(dataToPredict):
  nextPrices = model.predict(dataToPredict)
  nextPrices = scaler.inverse_transform(nextPrices)
  return nextPrices

Create a graph to compare with real prices

If we have created the logic that can predict the prices, we would like to know if the AI is close to reality.

To do that, we predict the prices, for the last month, and we will compare them with the real ones.

We need to create a matplotlib graph.

Let's start by creating a function named "createGraph".

def createGraph(nextPrices, stockPricesLastMonth):

Separate real prices

Next, we need to separate the "Open" prices from all of the last month's data.

The "Open" column is the second one in the table.

So we should use "iloc" method, and take all the values from that column.

Let's take a look at the example below.

real = stockPricesLastMonth.iloc[:, 1:2].values

Create a graph

In this step, we can create a matplotlib graph.

We will use green color for the real prices, and orange color for the predicted ones.

plt.plot(real, color = 'green', label = 'Real prices')
plt.plot(nextPrices, color = 'orange', label = 'Prices from ai')
plt.ylabel('Prices')
plt.xlabel('Timestamp')
plt.legend()
plt.show()

Predict

As we did in the previous lesson, we need to pack all of the logic into one function.

We will use the function called "predict" and put all of the functions that we built today.

def predict():
  stockPricesLastMonth, stockPricesTest = splitData(prices)
  dataToPredict = preparePredictData(stockPricesLastMonth, stockPricesTest)
  predicted = getPrediction(dataToPredict)
  createGraph(predicted, stockPricesLastMonth)