LSTM Interpretability: A Practical Tutorial

LSTMs are a staple for time series work, but they function as black boxes that offer zero intuition on why a specific prediction was made. If you're using Keras for forecasting, simply getting a low MSE isn't enough—you need to know if the model is actually picking up on seasonal trends or just over-relying on the most recent data point.

1. Data Preprocessing for Time Series

LSTMs require a 3D tensor input formatted as (samples, timesteps, features). To handle this, you have to transform flat temperature columns into overlapping windows.

import numpy as np
import pandas as pd
from sklearn.preprocessing import MinMaxScaler

# 1. Load data
df = pd.read_csv("weather_data.csv")
data = df['Temperature'].values.reshape(-1, 1)

# 2. Scale the data for stable neural network training
scaler = MinMaxScaler(feature_range=(0, 1))
scaled_data = scaler.fit_transform(data)

# 3. Create sequences: 7 days of lag to predict the 8th day
X, y = [], []
for i in range(7, len(scaled_data)):
 X.append(scaled_data[i-7:i])
 y.append(scaled_data[i])

X, y = np.array(X), np.array(y)
print(f"Input shape: {X.shape}") # Output: (Samples, 7, 1)

Skipping the MinMaxScaler is a common mistake; without it, LSTMs are highly susceptible to exploding gradients, which can crash your training process.

2. Model Architecture and Deployment

I use a stacked LSTM approach with dropout to prevent overfitting. A critical technical detail here is the return_sequences=True parameter on the first layer, which is mandatory when stacking LSTMs so the second layer receives the full sequence of hidden states.

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import LSTM, Dense, Dropout, Input
from tensorflow.keras.callbacks import EarlyStopping

# 1. Build the network
model = Sequential([
 Input(shape=(7, 1)),
 LSTM(units=100, activation='relu', return_sequences=True),
 Dropout(0.2),
 LSTM(units=100, activation='relu'),
 Dropout(0.2),
 Dense(units=1)
])

model.compile(optimizer='adam', loss='mse')

# 2. Configure Early Stopping
early_stopping = EarlyStopping(
 monitor='val_loss', 
 patience=10, 
 restore_best_weights=True
)

# 3. Train the model
history = model.fit(
 X, y, 
 epochs=50, 
 batch_size=32, 
 validation_split=0.2, 
 callbacks=[early_stopping]
)

3. Applying XAI Methods

To move beyond the black box, we can use permutation importance. This method identifies which "lag day" the model values most by shuffling the data for a specific day and measuring the resulting spike in error.

from sklearn.metrics import mean_squared_error

base_preds = model.predict(X, verbose=0)
base_error = mean_squared_error(y, base_preds)

feature_importance = []
for i in range(7):
 X_permuted = X.copy()
 np.random.shuffle(X_permuted[:, i, 0]) # Shuffle a specific lag day
 
 permuted_preds = model.predict(X_permuted, verbose=0)
 permuted_error = mean_squared_error(y, permuted_preds)
 
 feature_importance.append(permuted_error - base_error)

By calculating the difference between the permuted_error and base_error, you can pinpoint exactly which days in the 7-day window are driving the forecast. For a deeper dive into AI workflows, check out the resources at promptcube3.com.