Predicting Stock Prices with ChatGPT and EODHD

Unlock the power of AI and financial APIs — explore LSTM models, hyperparameter tuning, and real-world insights!

Predicting stock prices is a complex and detailed task that requires careful selection, construction, and evaluation of models. The process demands both patience and precision. However, the rise of advanced AI tools like ChatGPT has opened new possibilities for simplifying market predictions.

In this article, we’ll begin by obtaining time series data for the desired stock using the EODHD End-of-Day API. Next, we’ll leverage ChatGPT to handle the entire workflow, including data preparation, model building, prediction generation, and evaluation. Let’s get started!

Project Outline

After engaging in discussions with ChatGPT about various models and methodologies for stock price prediction, it provided an extensive range of options. To keep things simple, I chose one of the most popular methods: using an LSTM (Long Short-Term Memory) model.

I also asked ChatGPT to help draft an outline for a comprehensive Python project utilizing LSTM for stock price prediction. Here’s the refined outline it generated:

1. Obtaining Data
2. Linear Regression
3. A Basic LSTM Model
4. Improving the LSTM Model
5. Hyperparameter Tuning

While this wasn’t the bot’s initial suggestion, after refining it through multiple conversations, we arrived at a concise and straightforward structure. This article will follow the outlined steps, with all sections (except for obtaining data) entirely guided by ChatGPT.

1. Obtaining Data Using EODHD

To build an accurate ML model for stock price prediction, reliable and high-quality data is essential. EODHD provides a robust suite of financial APIs, offering a wide range of market data, including historical data, economic indicators, and financial news.

For this project, we’ll predict Microsoft’s stock prices using its time series data over a long period. We’ve chosen EODHD’s End-of-Day API to extract Microsoft’s historical data from 2010 to the present. Below is the Python code used for this purpose:

def get_historical_data(symbol, start, end):
    api_key = 'YOUR API KEY'
    api_response = requests.get(f'https://eodhistoricaldata.com/api/eod/{symbol}?api_token={api_key}&fmt=json&from={start}&to={end}').json()
    df = pd.DataFrame(api_response).drop('close', axis = 1)
    df.columns = ['Date', 'Open', 'High', 'Low', 'Close', 'Volume']
    df.Date = pd.to_datetime(df.Date)
    return df

df = get_historical_data('MSFT', '2010-01-01', '2023-07-23')
df.to_csv('msft.csv')

df.tail()

Explanation:

• Replace YOUR API KEY with your EODHD API key, which you can obtain after creating an account.
• The data is saved as a CSV file (msft.csv) to avoid repeated API calls during subsequent processes.
• While the API provides over 30 years of data, we’ve limited it to 13 years as this is sufficient for our program.

Below is the resulting dataframe, which will serve as the input for our stock price prediction model:

Microsoft stock (MSFT) Historical Data

2. Linear Regression

Before diving into building an LSTM model, we opted to start with a basic Linear Regression model for stock price prediction. This approach allows us to observe the results and understand the data better before implementing a more complex model.

Using ChatGPT, we requested a Python program to build a simple Linear Regression model with Scikit-Learn. The bot provided a clear set of steps to construct the regression model, followed by this sample code:

import pandas as pd
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import r2_score, mean_squared_error
import matplotlib.pyplot as plt

# Get the stock data
df = pd.read_csv('msft.csv').dropna()
df = df.set_index('Date')

# Add a column for the target variable
df['Price'] = df['Close']

# Define the features and target variable
features = ['Open', 'High', 'Low', 'Volume']
target = 'Price'

# Split the data into training and testing sets
train_size = 0.8
train_data = df[:int(train_size * len(df))]
test_data = df[int(train_size * len(df)):]

# Normalize the data
scaler = StandardScaler()
train_data[features] = scaler.fit_transform(train_data[features])
test_data[features] = scaler.transform(test_data[features])

# Train the model
model = LinearRegression()
model.fit(train_data[features], train_data[target])

# Make predictions on the test data
predictions = model.predict(test_data[features])

# Compute the R^2 score and RMSE
r2 = r2_score(test_data[target], predictions)
rmse = np.sqrt(mean_squared_error(test_data[target], predictions))

print(f'R^2 score: {r2:.4f}')
print(f'RMSE: {rmse:.4f}')

# Plot the predicted and actual values
plt.plot(test_data[target].values, label='Actual')
plt.plot(predictions, label='Predicted')
plt.legend()
plt.show()

This code is exceptionally well-structured and follows all standard practices. One of its standout features is the inclusion of detailed comments for each step, making it easy to understand the process and logic. Additionally, ChatGPT provided a concise explanation summarizing the entire workflow.

One minor adjustment I’d consider for this code is experimenting with the model parameters to determine the most suitable configuration for our specific use case. However, this is a topic for another time, as the primary goal here is to understand ChatGPT’s code and evaluate its results.

3. A Basic LSTM Model

To take our stock price prediction to the next level, we asked ChatGPT to generate a Python program for building an LSTM (Long Short-Term Memory) model. LSTM is a powerful neural network architecture designed for analyzing sequential data by selectively retaining or forgetting information. This makes it especially suitable for tasks like time-series analysis, speech recognition, and natural language processing.

Below is the code generated by ChatGPT to build the predictive LSTM model:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.metrics import *
from sklearn.preprocessing import MinMaxScaler
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense, Dropout

# Load the stock price data
df = pd.read_csv('msft.csv').dropna()
df = df.set_index('Date')

# Preprocess the data
scaler = MinMaxScaler()
df_scaled = scaler.fit_transform(df['Close'].values.reshape(-1, 1))

# Define the training and testing data
train_size = int(len(df_scaled) * 0.8)
train_data = df_scaled[:train_size, :]
test_data = df_scaled[train_size:, :]

# Define the function to create a sliding window dataset
def create_dataset(dataset, time_steps=1):
    X_data, y_data = [], []
    for i in range(len(dataset)-time_steps-1):
        X_data.append(dataset[i:(i+time_steps), 0])
        y_data.append(dataset[i + time_steps, 0])
    return np.array(X_data), np.array(y_data)

# Define the time steps and create the sliding window dataset
time_steps = 60
X_train, y_train = create_dataset(train_data, time_steps)
X_test, y_test = create_dataset(test_data, time_steps)

# Reshape the data for LSTM input
X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))
X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))

# Define the LSTM model architecture
model = Sequential()
model.add(LSTM(units=64, return_sequences=True, input_shape=(X_train.shape[1], 1)))
model.add(Dropout(0.2))
model.add(LSTM(units=64, return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(units=64, return_sequences=False))
model.add(Dropout(0.2))
model.add(Dense(units=1))

# Compile the model
model.compile(optimizer='adam', loss='mean_squared_error')

# Train the model
model.fit(X_train, y_train, epochs=50, batch_size=64, validation_data=(X_test, y_test), verbose=1)

# Make predictions
y_pred = model.predict(X_test)

# Inverse transform the predicted and actual values
y_pred = scaler.inverse_transform(y_pred)
y_test = y_test.reshape(y_pred.shape[0], 1)
y_test = scaler.inverse_transform(y_test)

# Evaluate the model
mse = mean_squared_error(y_test, y_pred)
msle = mean_squared_log_error(y_test, y_pred)
mae = mean_absolute_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)

print('MSE: ', mse)
print('MSLE: ', msle)
print('MAE: ', mae)
print('R-squared: ', r2)

# Plot the predicted vs actual values
plt.figure(figsize=(10, 6))
plt.style.use('fivethirtyeight')
plt.plot(y_test, label='Actual', linewidth = 3, alpha = 0.4)
plt.plot(y_pred, label='Predicted', linewidth = 1.5)
plt.xlabel('Days')
plt.ylabel('Stock Price')
plt.title('LSTM: Actual vs Predicted')
plt.legend()
plt.show()

This code is remarkably simple, clear, and well-structured, with detailed comments for every step, making it easy to follow and understand. ChatGPT provided a straightforward implementation of TensorFlow to build an LSTM model, which aligns perfectly with the request. The evaluation process, complemented by a graph, adds significant value. While I customized the plot slightly to enhance its appearance, I left the rest of the code untouched. Here’s the result:

LTSM: Actual vs Predicted

4. Improving the LSTM Model

The LSTM model we previously built and used for predictions is quite basic and can be enhanced in multiple ways. To explore improvements, I asked ChatGPT for suggestions to optimize the code for a better-performing LSTM model. Here’s what it suggested:

1. Increase the Number of LSTM Layers: Adding more LSTM layers helps the model capture complex patterns in the data.
2. Increase the Number of Neurons: Expanding the number of neurons in each LSTM layer enables the model to learn more intricate relationships between features and targets.
3. Adjust the Learning Rate: Fine-tuning the learning rate can strike a balance between slow training and overshooting optimal weights.
4. Add Dropout Layers: Incorporating dropout layers reduces overfitting by randomly disabling neurons during training, improving generalization.
5. Experiment with Activation Functions: Testing different activation functions like tanh, relu, or sigmoid can enhance the model's ability to capture relationships in the data.
6. Use More Data: Expanding the dataset, whether by increasing the historical range or adding features like news sentiment or technical indicators, can significantly improve model performance.
7. Use a Different Loss Function: Opting for loss functions better suited to time series data, such as Mean Squared Logarithmic Error (MSLE), can improve accuracy.

ChatGPT provided a detailed guideline for improving AI models, particularly LSTMs. When asked to incorporate these enhancements into the previous code, it returned an updated version of the model, implementing these suggestions:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.metrics import *
from sklearn.preprocessing import MinMaxScaler
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense, Dropout
from tensorflow.keras.callbacks import EarlyStopping

# Obtain stock data for Microsoft
df = pd.read_csv('msft.csv').dropna()

# Extract the closing prices
close_prices = df[["Date", "Close"]].copy()
close_prices.columns = ["ds", "y"]
close_prices.set_index("ds", inplace=True)

# Normalize the data
scaler = MinMaxScaler(feature_range=(0, 1))
close_prices_scaled = scaler.fit_transform(close_prices)

# Create training and testing data
train_size = int(len(close_prices) * 0.8)
train_data = close_prices_scaled[:train_size]
test_data = close_prices_scaled[train_size:]

# Define the number of time steps to look back
look_back = 60

# Create training and testing data for the LSTM model
def create_data(data):
    X, Y = [], []
    for i in range(look_back, len(data)):
        X.append(data[i - look_back : i, 0])
        Y.append(data[i, 0])
    X, Y = np.array(X), np.array(Y)
    X = np.reshape(X, (X.shape[0], X.shape[1], 1))
    return X, Y

X_train, Y_train = create_data(train_data)
X_test, Y_test = create_data(test_data)

# Define the LSTM model
model = Sequential()

model.add(LSTM(units=64, return_sequences=True, input_shape=(X_train.shape[1], 1)))
model.add(Dropout(0.2))

model.add(LSTM(units=128, return_sequences=True))
model.add(Dropout(0.2))

model.add(LSTM(units=64))
model.add(Dropout(0.2))

model.add(Dense(units=1))

# Compile the model
model.compile(optimizer="adam", loss="mean_squared_error")

# Define the early stopping criteria
early_stop = EarlyStopping(monitor="val_loss", patience=10)

# Fit the model to the training data
history = model.fit(X_train, Y_train, epochs=100, batch_size=32, validation_data=(X_test, Y_test), callbacks=[early_stop])

# Make predictions on the test data
y_pred = model.predict(X_test)

# Inverse the scaling of the data
y_pred = scaler.inverse_transform(y_pred)
Y_test = scaler.inverse_transform([Y_test])

# Evaluate the model
mse = mean_squared_error(Y_test[0], y_pred)
msle = mean_squared_log_error(Y_test[0], y_pred)
mae = mean_absolute_error(Y_test[0], y_pred)
r2 = r2_score(Y_test[0], y_pred)

print('MSE: ', mse)
print('MSLE: ', msle)
print('MAE: ', mae)
print('R-squared: ', r2)

# Plot the predicted vs actual values
plt.figure(figsize=(10, 6))
plt.style.use('fivethirtyeight')
plt.plot(Y_test.flatten(), label="Actual", linewidth = 3, alpha = 0.4)
plt.plot(y_pred.flatten(), label="Predicted", linewidth = 1.5)
plt.title("MSFT Stock Price Prediction")
plt.xlabel("Days")
plt.ylabel("Stock Price")
plt.legend()
plt.show()

This updated code can be seen as an extended version of the previous implementation, incorporating the refinements suggested by ChatGPT. As per ChatGPT’s description, the modifications include:

• Increasing the Number of LSTM Layers
• Increasing the Number of Neurons in Each Layer
• Adding Dropout Layers
• Adjusting the Learning Rate
• Training on the Complete Historical Dataset
• Switching to Mean Squared Logarithmic Error (MSLE) for better suitability to time series data

In my view, while the changes are not dramatic, they are effective. Substantial improvements to a model don’t always require sweeping modifications — small, focused adjustments can significantly enhance performance.

Previous model:

previous model

Improved model:

improved model

The improvements are evident:

• R² Score: Increased from 0.61 to 0.91, indicating much higher predictive accuracy.
• Error Metrics (MSE, MSLE, MAE): All show a significant reduction, moving closer to zero, which reflects a better-fitting model.

The enhanced model demonstrates clear improvements in performance. ChatGPT’s suggestions, even with incremental adjustments, effectively boosted the model’s accuracy and reliability.

5. Hyperparameter Tuning

No AI or machine learning project is truly complete without hyperparameter tuning. Here’s a brief overview of the process:

Hyperparameter Tuning refers to optimizing the settings of a machine learning model to enhance its performance. These settings, or hyperparameters, are predefined before the training process and can include factors like the learning rate, number of layers, neurons, batch size, and more. The goal is to find the combination of parameters that yields the best predictive accuracy.

When prompted to perform hyperparameter tuning on the previously generated model, ChatGPT produced the following code:

import numpy as np
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, LSTM, Dropout
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.wrappers.scikit_learn import KerasRegressor
from sklearn.model_selection import GridSearchCV, TimeSeriesSplit

# Download the data
df = pd.read_csv('msft.csv').dropna()
df = df.set_index('Date')

# Scale the data
scaler = MinMaxScaler()
scaled_data = scaler.fit_transform(df['Close'].values.reshape(-1, 1))

# Define the training and test sets
train_size = int(len(scaled_data) * 0.8)
train_data = scaled_data[:train_size]
test_data = scaled_data[train_size:]

# Define the function to create the LSTM model
def create_model(neurons=50, layers=2, dropout=0.2, learning_rate=0.001):
    model = Sequential()
    for i in range(layers):
        model.add(LSTM(units=neurons, return_sequences=True, input_shape=(train_data.shape[1], 1)))
        model.add(Dropout(dropout))
    model.add(LSTM(units=neurons))
    model.add(Dropout(dropout))
    model.add(Dense(units=1))
    optimizer = Adam(learning_rate=learning_rate)
    model.compile(optimizer=optimizer, loss='mean_squared_error')
    return model

# Define the parameter grid for hyperparameter tuning
param_grid = {
    'neurons': [50, 100, 200],
    'layers': [2, 3],
    'dropout': [0.2, 0.3],
    'learning_rate': [0.001, 0.01]
}

# Define the time series split for cross-validation
tscv = TimeSeriesSplit(n_splits=3)

# Create the KerasRegressor for hyperparameter tuning
model = KerasRegressor(build_fn=create_model, epochs=50, batch_size=32, verbose=0)

# Perform the hyperparameter tuning
grid = GridSearchCV(estimator=model, param_grid=param_grid, cv=tscv, scoring='neg_mean_squared_error', n_jobs=-1)
grid_result = grid.fit(train_data, train_data)

# Print the best parameters and score
print("Best parameters: ", grid_result.best_params_)
print("Best score: ", np.sqrt(-grid_result.best_score_))

# Use the best model to make predictions on the test data
best_model = grid_result.best_estimator_.model
predictions = best_model.predict(test_data.reshape(-1, 1))
predictions = scaler.inverse_transform(predictions)

# Plot the predictions and actual values
import matplotlib.pyplot as plt

# Plot the predicted vs actual values
plt.figure(figsize=(10, 6))
plt.style.use('fivethirtyeight')
plt.plot(pd.to_datetime(df.index[train_size:]), df['Close'][train_size:], label='Actual', linewidth = 3, alpha = 0.4)
plt.plot(pd.to_datetime(df.index[train_size:]), predictions, label='Predicted', linewidth = 1.5)
plt.title(f"MSFT Stock Price Prediction")
plt.xlabel("Date")
plt.ylabel("Stock Price")
plt.legend()
plt.show()

As with the previously generated programs, this code maintains a clean structure, includes neatly written comments, and follows the standard procedure for hyperparameter tuning. It defines a param_grid and utilizes the GridSearchCV function to perform a comprehensive grid search, identifying the optimal model parameters.

Best Parameters and Score:

• Best Parameters: {'dropout': 0.2, 'layers': 2, 'learning_rate': 0.001, 'neurons': 100}
• Best Score: 0.0047933462470134474

After identifying the best parameters, ChatGPT proceeded to utilize them in building a refined model and making predictions. The resulting graph illustrates the performance of the optimized model after hyperparameter tuning:

Microsoft stock (MSFT) Price Prediction with LTSM

Final Thoughts

While ChatGPT provided valuable assistance in generating impressive code for predicting stock prices, the models often fall short in real-world market conditions. This is largely because stock prices are influenced by emotions and sentiments — factors that cannot be fully captured or quantified by AI-generated code.

However, the work done here is far from meaningless. Instead, it serves as a foundation for deeper analysis. These AI models offer a glimpse into potential price movements and help establish a framework for further investigation. Predictive analytics should be considered a supplementary tool rather than a standalone solution when making investment decisions.

A special thanks to EODHD for offering an excellent repository of financial APIs. Their precise and comprehensive data significantly enhanced the model’s performance, making it an invaluable resource for projects like this. We highly recommend exploring their platform, as it can be a game-changer for future projects and research endeavors.

Thank you for reading, and we hope this article provided useful insights!

Stay tuned for more valuable articles that aim to enhance your data science skills and market analysis capabilities.

The original article was published in the EODHD Academy by Nikhil Adithyan

Please note that this article is for informational purposes only and should not be taken as financial advice. We do not bear responsibility for any trading decisions made based on the content of this article. Readers are advised to conduct their research or consult with a qualified financial professional before making any investment decisions.

For those eager to delve deeper into such insightful articles and broaden their understanding of different strategies in financial markets, we invite you to follow our account and subscribe for email notifications.