R4-LSTM学习笔记

• 🍨 本文为🔗365天深度学习训练营 中的学习记录博客
• 🍖 原作者：K同学啊

导入数据

import tensorflow as tf
import pandas     as pd
import numpy      as npgpus = tf.config.list_physical_devices("GPU")
if gpus:tf.config.experimental.set_memory_growth(gpus[0], True)  #设置GPU显存用量按需使用tf.config.set_visible_devices([gpus[0]],"GPU")
print(gpus)df_1 = pd.read_csv(r"C:\Users\11054\Desktop\kLearning\R4\woodpine2.csv")

[]

数据可视化

import matplotlib.pyplot as plt
import seaborn as snsplt.rcParams['savefig.dpi'] = 500 #图片像素
plt.rcParams['figure.dpi']  = 500 #分辨率fig, ax =plt.subplots(1,3,constrained_layout=True, figsize=(14, 3))sns.lineplot(data=df_1["Tem1"], ax=ax[0])
sns.lineplot(data=df_1["CO 1"], ax=ax[1])
sns.lineplot(data=df_1["Soot 1"], ax=ax[2])
plt.show()

#构建数据集

dataFrame = df_1.iloc[:,1:]
dataFrame

	Tem1	CO 1	Soot 1
0	25.0	0.000000	0.000000
1	25.0	0.000000	0.000000
2	25.0	0.000000	0.000000
3	25.0	0.000000	0.000000
4	25.0	0.000000	0.000000
...	...	...	...
5943	295.0	0.000077	0.000496
5944	294.0	0.000077	0.000494
5945	292.0	0.000077	0.000491
5946	291.0	0.000076	0.000489
5947	290.0	0.000076	0.000487

5948 rows × 3 columns

设置X、y

# 取前8个时间段的Tem1、CO 1、Soot 1为X，第9个时间段的Tem1为y。
width_X = 8
width_y = 1
X = []
y = []in_start = 0for _, _ in df_1.iterrows():in_end  = in_start + width_Xout_end = in_end   + width_yif out_end < len(dataFrame):X_ = np.array(dataFrame.iloc[in_start:in_end , ])X_ = X_.reshape((len(X_)*3))y_ = np.array(dataFrame.iloc[in_end  :out_end, 0])X.append(X_)y.append(y_)in_start += 1X = np.array(X)
y = np.array(y)X.shape, y.shape

((5939, 24), (5939, 1))

# 归一化
from sklearn.preprocessing import MinMaxScaler#将数据归一化，范围是0到1
sc       = MinMaxScaler(feature_range=(0, 1))
X_scaled = sc.fit_transform(X)
X_scaled.shape

(5939, 24)

X_scaled = X_scaled.reshape(len(X_scaled),width_X,3)
X_scaled.shape

(5939, 8, 3)

# 划分数据集
X_train = np.array(X_scaled[:5000]).astype('float64')
y_train = np.array(y[:5000]).astype('float64')X_test  = np.array(X_scaled[5000:]).astype('float64')
y_test  = np.array(y[5000:]).astype('float64')
X_train.shape

(5000, 8, 3)

构建模型

# 多层 LSTM
import tensorflow.python.keras as keras
from tensorflow.python.keras.layers.core import Activation, Dropout, Dense
from tensorflow.python.keras.layers import Flatten, LSTM
model_lstm = keras.Sequential()
model_lstm.add(LSTM(units=64, activation='relu', return_sequences=True,input_shape=(X_train.shape[1], 3)))
model_lstm.add(LSTM(units=64, activation='relu'))model_lstm.add(Dense(width_y))

# 只观测loss数值，不观测准确率，所以删去metrics选项
from tensorflow.python.keras.optimizers import adam_v2
optimizer = adam_v2.Adam(1e-3)
model_lstm.compile(optimizer=optimizer,loss='mean_squared_error')  # 损失函数用均方误差

X_train.shape, y_train.shape

((5000, 8, 3), (5000, 1))

# 注意 此处函数弃用 需要修改源码
#‘tensorflow.python.distribute.input_lib‘ has no attribute 
# DistributedDataset 实际是判断是否为分布式数据，返回值直接改为False
history_lstm = model_lstm.fit(X_train, y_train,batch_size=64,epochs=40,validation_data=(X_test, y_test),validation_freq=1)

Epoch 1/40
79/79 [==============================] - 5s 18ms/step - loss: 11953.3770 - val_loss: 4300.0811
Epoch 2/40
79/79 [==============================] - 1s 14ms/step - loss: 154.9082 - val_loss: 996.6027
Epoch 3/40
79/79 [==============================] - 1s 13ms/step - loss: 15.5335 - val_loss: 341.1582
Epoch 4/40
79/79 [==============================] - 1s 13ms/step - loss: 7.5403 - val_loss: 290.5845
Epoch 5/40
79/79 [==============================] - 1s 12ms/step - loss: 7.3599 - val_loss: 265.5993
Epoch 6/40
79/79 [==============================] - 1s 13ms/step - loss: 7.0518 - val_loss: 257.2565
Epoch 7/40
79/79 [==============================] - 1s 14ms/step - loss: 7.2212 - val_loss: 291.9721
Epoch 8/40
79/79 [==============================] - 1s 14ms/step - loss: 6.5968 - val_loss: 214.1582
Epoch 9/40
79/79 [==============================] - 1s 12ms/step - loss: 7.5128 - val_loss: 243.2180
Epoch 10/40
79/79 [==============================] - 1s 14ms/step - loss: 6.6279 - val_loss: 218.3901
Epoch 11/40
79/79 [==============================] - 1s 14ms/step - loss: 6.3576 - val_loss: 220.4561
Epoch 12/40
79/79 [==============================] - 1s 13ms/step - loss: 6.5356 - val_loss: 243.2316
Epoch 13/40
79/79 [==============================] - 1s 12ms/step - loss: 6.8603 - val_loss: 225.0939
Epoch 14/40
79/79 [==============================] - 1s 13ms/step - loss: 7.3385 - val_loss: 154.1982
Epoch 15/40
79/79 [==============================] - 1s 14ms/step - loss: 7.1614 - val_loss: 225.3159
Epoch 16/40
79/79 [==============================] - 1s 13ms/step - loss: 6.5654 - val_loss: 199.5660
Epoch 17/40
79/79 [==============================] - 1s 12ms/step - loss: 6.4847 - val_loss: 176.0666
Epoch 18/40
79/79 [==============================] - 1s 11ms/step - loss: 6.3168 - val_loss: 217.2618
Epoch 19/40
79/79 [==============================] - 1s 12ms/step - loss: 6.6176 - val_loss: 244.9601
Epoch 20/40
79/79 [==============================] - 1s 13ms/step - loss: 6.9601 - val_loss: 229.4581
Epoch 21/40
79/79 [==============================] - 1s 15ms/step - loss: 5.8222 - val_loss: 351.2197
Epoch 22/40
79/79 [==============================] - 1s 13ms/step - loss: 7.2396 - val_loss: 149.9563
Epoch 23/40
79/79 [==============================] - 1s 13ms/step - loss: 7.0510 - val_loss: 273.4801
Epoch 24/40
79/79 [==============================] - 1s 14ms/step - loss: 6.3674 - val_loss: 254.2635
Epoch 25/40
79/79 [==============================] - 1s 13ms/step - loss: 6.6236 - val_loss: 139.3550
Epoch 26/40
79/79 [==============================] - 1s 14ms/step - loss: 5.7532 - val_loss: 296.6612
Epoch 27/40
79/79 [==============================] - 1s 14ms/step - loss: 6.8470 - val_loss: 305.5312
Epoch 28/40
79/79 [==============================] - 1s 13ms/step - loss: 7.1153 - val_loss: 160.2791
Epoch 29/40
79/79 [==============================] - 1s 12ms/step - loss: 5.9563 - val_loss: 235.7691
Epoch 30/40
79/79 [==============================] - 1s 13ms/step - loss: 7.2391 - val_loss: 168.0048
Epoch 31/40
79/79 [==============================] - 1s 13ms/step - loss: 5.8283 - val_loss: 197.9875
Epoch 32/40
79/79 [==============================] - 1s 14ms/step - loss: 5.3628 - val_loss: 279.9405
Epoch 33/40
79/79 [==============================] - 1s 14ms/step - loss: 7.0928 - val_loss: 134.6513
Epoch 34/40
79/79 [==============================] - 1s 14ms/step - loss: 6.9159 - val_loss: 213.4102
Epoch 35/40
79/79 [==============================] - 1s 14ms/step - loss: 5.9036 - val_loss: 190.3418
Epoch 36/40
79/79 [==============================] - 1s 14ms/step - loss: 7.2137 - val_loss: 136.0768
Epoch 37/40
79/79 [==============================] - 1s 14ms/step - loss: 5.9089 - val_loss: 176.8896
Epoch 38/40
79/79 [==============================] - 1s 13ms/step - loss: 6.0480 - val_loss: 153.8418
Epoch 39/40
79/79 [==============================] - 1s 12ms/step - loss: 5.7332 - val_loss: 255.8892
Epoch 40/40
79/79 [==============================] - 1s 14ms/step - loss: 6.2338 - val_loss: 163.2283

# 支持中文
plt.rcParams['font.sans-serif'] = ['SimHei']  # 用来正常显示中文标签
plt.rcParams['axes.unicode_minus'] = False  # 用来正常显示负号plt.figure(figsize=(5, 3),dpi=120)plt.plot(history_lstm.history['loss']    , label='LSTM Training Loss')
plt.plot(history_lstm.history['val_loss'], label='LSTM Validation Loss')plt.title('Training and Validation Loss')
plt.legend()
plt.show()

调用模型

predicted_y_lstm = model_lstm.predict(X_test)                        # 测试集输入模型进行预测y_test_one = [i[0] for i in y_test]
predicted_y_lstm_one = [i[0] for i in predicted_y_lstm]plt.figure(figsize=(5, 3),dpi=120)
# 画出真实数据和预测数据的对比曲线
plt.plot(y_test_one[:1000], color='red', label='真实值')
plt.plot(predicted_y_lstm_one[:1000], color='blue', label='预测值')plt.title('Title')
plt.xlabel('X')
plt.ylabel('Y')
plt.legend()
plt.show()

from sklearn import metrics
"""
RMSE ：均方根误差  ----->  对均方误差开方
R2   ：决定系数，可以简单理解为反映模型拟合优度的重要的统计量
"""
RMSE_lstm  = metrics.mean_squared_error(predicted_y_lstm, y_test)**0.5
R2_lstm    = metrics.r2_score(predicted_y_lstm, y_test)print('均方根误差: %.5f' % RMSE_lstm)
print('R2: %.5f' % R2_lstm)

均方根误差: 12.77608
R2: 0.70679

个人总结

LSTM 的基本结构

LSTM 的核心思想是通过门控机制来控制信息的流动。一个标准的 LSTM 单元包含以下三个门结构：

1. 遗忘门（Forget Gate）：

   • 决定哪些信息需要从细胞状态中丢弃。
   • 计算公式：KaTeX parse error: Can't use function '\(' in math mode at position 1: \̲(̲ f_t = \sigma(W…
   • 其中 KaTeX parse error: Can't use function '\(' in math mode at position 1: \̲(̲ \sigma \) 是 Sigmoid 激活函数，KaTeX parse error: Can't use function '\(' in math mode at position 1: \̲(̲ W_f \) 是权重矩阵，KaTeX parse error: Can't use function '\(' in math mode at position 1: \̲(̲ b_f \) 是偏置项，KaTeX parse error: Can't use function '\(' in math mode at position 1: \̲(̲ h_{t-1} \) 是前一时间步的隐藏状态，KaTeX parse error: Can't use function '\(' in math mode at position 1: \̲(̲ x_t \) 是当前时间步的输入。

2. 输入门（Input Gate）：

   • 决定哪些新信息需要添加到细胞状态中。
   • 计算公式：KaTeX parse error: Can't use function '\(' in math mode at position 1: \̲(̲ i_t = \sigma(W…
   • 以及新候选值：KaTeX parse error: Can't use function '\(' in math mode at position 1: \̲(̲ \tilde{C_t} = …

3. 输出门（Output Gate）：

   • 决定当前细胞状态的哪些部分将输出。
   • 计算公式：KaTeX parse error: Can't use function '\(' in math mode at position 1: \̲(̲ o_t = \sigma(W…

细胞状态（Cell State）

细胞状态 KaTeX parse error: Can't use function '\(' in math mode at position 1: \̲(̲ C_t \)是 LSTM 的核心，它通过门控机制来传递长期依赖信息。

• 更新细胞状态：

  • KaTeX parse error: Can't use function '\(' in math mode at position 1: \̲(̲ C_t = f_t \odo…
  • 其中 KaTeX parse error: Can't use function '\(' in math mode at position 1: \̲(̲ \odot \) 表示逐元素乘法。

• 计算当前时间步的隐藏状态：

  • KaTeX parse error: Can't use function '\(' in math mode at position 1: \̲(̲ h_t = o_t \odo…

LSTM 的优点

1. 处理长期依赖问题：通过门控机制，LSTM 能够有效处理序列数据中的长期依赖问题，避免了传统 RNN 的梯度消失和梯度爆炸问题。

2. 灵活性：LSTM 可以用于各种序列学习任务，如语言模型、语音识别、时间序列预测等。

3. 性能：在许多序列学习任务中，LSTM 表现出色，尤其是在需要记忆长期信息的场景中。