利用PyTorch深度学习框架进行多元回归

前言

大多数数据科学家以往经常常是利用传统的机器学习框架sklearn搭建多元回归模型，随着深度学习和强化学习越来越普及，很多数据科学家尝试使用深度学习框架来进行多元回归实验，下面来做一个利用PyTorch深度学习框架做一个完整的多元回归数据科学实验。从该实验，你将学到一下内容，并通过这一篇博客成功运用深度学习框架PyTorch进行多元回归模型搭建。

• 如何加载数据
• 如何转换数据
• 如何定义模型
• 如何评估模型
• 如何验收模型
• 如何保存模型和加载模型

一个典型的PyTorch建模过程往往包含了一下几个过程

接下来，我们就严格按照上面步骤来一步一步进行。

数据加载

数据加载是指将将本地或者服务器，云端的数据读取进来，并进行必要的预处理，包括填充空值，去掉异常值，编码，格式转换，数据分裂成训练集和测试集，这一步骤可以利用我们熟悉的pandas库处理。

store = pd.read_csv(r"D:\项目\商简智能\回归预测题目\store.csv")
train = pd.read_csv(r"D:\项目\商简智能\回归预测题目\train.csv")
train_df = pd.merge(left=train, right=store, left_on='商店ID', right_on='商店ID', how='left')
train_df = train_df.query("周销量>0")
train_df.dropna(how='any', inplace=True)
train_df['商店ID'] = train_df['商店ID'].astype('float64')
train_df['年'] = train_df['年'].astype('float64')
train_df['周'] = train_df['周'].astype('float64')
train_df['节日A'] = train_df['节日A'].astype('float64')
train_df['节日B'] = train_df['节日B'].astype('float64')
train_df['节日C'] = train_df['节日C'].astype('float64')
train_df['商店模式'] = LabelEncoder() .fit_transform(train_df['商店模式'] ) #编码
train_df['商店级别'] = LabelEncoder() .fit_transform(train_df['商店级别'] ) #编码
train_df = train_df[["商店ID", "年",  "周",  "营业天数",  "打折天数",  "非节日",  "节日A",  "节日B",  "节日C",  "商店模式",  "商店级别",  "竞争者最近距离",  "周销量"]]
X, y =  train_df.drop(['周销量'], axis=1), train_df.周销量 #特征和目标
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3, shuffle=True, random_state=0) #划分训练测试集
print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)

通过上面代码可以看到本次实验是读取本地数据，具体数据格式和解读参见这篇文章，并将数值型变量统一转化为float64型，将类别型变量编码，定义出特征列和标签列，并利用train_test_split将数据集分裂成训练集和测试集，可以打印出训练集和测试集的形状大小以检验数据分割的正确与否

(51300, 12) (21987, 12) (51300,) (21987,)

数据转换

经过数据加载步骤后，数据没有刺头了变得规规矩矩了，哪一块就是哪一块，但是都是numpy，series，dataframe等数据格式，而深度学习框架只认识张量(tensor)数据格式，在PyTorch的世界，不管是模型的输入，输出还是模型的参数都是由tensor来编码的，所以我们需要将之前得到的dataframe转化为tensor，并适当的分组(batch)以便输入模型进行小批量(minibatch)训练，PyTorch提供了两种原生的数据处理方法torch.utils.data.DataLoader 和 torch.utils.data.Dataset，Dataset 使得你可以使用一些预置数据和自己的数据，经过Dataset的数据存储了样本和其对应的标签，而DataLoader 可以把Dataset的样本打包成一个一个小组形成迭代器，使得在模型训练的时候可以只对这些迭代器进行循环训练。

X_train, X_test = X_train.values, X_test.values
y_train, y_test = y_train.values.reshape(-1, 1), y_test.values.reshape(-1, 1)
X_train  = torch.from_numpy(X_train).type(torch.FloatTensor)
X_test = torch.from_numpy(X_test).type(torch.FloatTensor)
y_train = torch.from_numpy(y_train).type(torch.FloatTensor)
y_test = torch.from_numpy(y_test).type(torch.FloatTensor)
training_data = TensorDataset(X_train, y_train)
test_data = TensorDataset(X_test, y_test)
train_dataloader = DataLoader(training_data, batch_size=64)
test_dataloader = DataLoader(test_data, batch_size=64)

模型定义

模型定义是指定义好深度学习的网络结构，俗称神经网络，神经网络由一些模型和层构成，数据在这些结构上进行各种各样的运算，在PyTorch里，每一个模型都继承 nn.Module，我们可以搭建和管理各种风格和各种样子的神经网络。

class Net(nn.Module):      def __init__(self, features):super(Net, self).__init__() self.h1 = nn.Linear(features, 30, bias= True)self.a1 = nn.ReLU()self.h2 = nn.Linear(30, 10)self.a2 = nn.ReLU()self.regression = nn.Linear(10, 1)def forward(self, x): x = self.h1(x)x = self.a1(x)x = self.h2(x)x = self.a2(x)y_pred = self.regression(x)return y_pred

通过上面代码，我们搭建了一个名为Net的神经网络，该网络继承了nn.Module，并定义了初始化化和向前传播路径，网络由2个Linear层和2个ReLU激活函数组成，输入层的参数是特征数(features)30，隐藏层是3010，输出层是10*1，最后返回一个预测值，我们可以实例化该网络，打印出网络结构

Net((h1): Linear(in_features=12, out_features=30, bias=True)(a1): ReLU()(h2): Linear(in_features=30, out_features=10, bias=True)(a2): ReLU()(regression): Linear(in_features=10, out_features=1, bias=True)
)

模型训练

深度学习的模型训练首先是向前传播得到预测值，其次，在给定的损失函数前提下计算损失的梯度，不断更新模型里面的参数，同时可以记录训练集的损失函数值和测试集的损失函数值以便后续可视化两者变化曲线。

train_loss = []
test_loss = []epochs = 201
for epoch in range(epochs):model.train()for xb, yb in train_dataloader:pred = model(xb)loss = criterion(pred, yb)loss.requires_grad_(True) optimizer.zero_grad()loss.backward()optimizer.step()if epoch%10==0:model.eval()with torch.no_grad():train_epoch_loss = sum(criterion(model(xb), yb) for xb, yb in train_dataloader)test_epoch_loss = sum(criterion(model(xb), yb) for xb, yb in test_dataloader)train_loss.append(train_epoch_loss.data.item() / len(train_dataloader))test_loss.append(test_epoch_loss.data.item() / len(test_dataloader))template = ("epoch:{:2d}, 训练损失:{:.5f}, 验证损失:{:.5f}")print(template.format(epoch, train_epoch_loss.data.item() / len(train_dataloader), test_epoch_loss.data.item() / len(test_dataloader)))
print('训练完成')

模型评估

模型评估是指对训练好的模型进行某些指标的评估，由于在模型训练的时候我们保存了训练集和测试集的损失值，可以在这里直接画出来。

fig = plt.figure(figsize = (6,4))
plt.plot(range(len(train_loss)), train_loss, label = "train_loss")
plt.plot(range(len(test_loss)), test_loss, label = "test_loss")
plt.legend()
plt.show()

模型保存与加载

辛辛苦苦训练出来并通过相关指标评估的模型，自然希望能够保存下来以便下次能够在此利用，PyTorch提供2中模型保存和加载策略，一个是只存储模型中的参数，该方法速度快，占用空间少（官方推荐使用），一个是存储整个模型，这里采用前者。

torch.save(model.state_dict(), ".\model_parameter.pth") #保存模型parameter = torch.load(".\model_parameter.pth") #加载模型
model.load_state_dict(parameter)

完整代码

# -*- encoding: utf-8 -*-
'''
@Project :   sales
@Desc    :   基于pytorch的深度学习销量预测
@Time    :   2023/02/07 16:30:14
@Author  :   帅帅de三叔,[e-mail]
'''import math
import torch
from torch import nn
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from torch.utils.data import TensorDataset
from torch.utils.data import DataLoader
from sklearn.preprocessing import LabelEncoder
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler # 数据标准化store = pd.read_csv(r"D:\项目\商简智能\回归预测题目\store.csv")
train = pd.read_csv(r"D:\项目\商简智能\回归预测题目\train.csv")
train_df = pd.merge(left=train, right=store, left_on='商店ID', right_on='商店ID', how='left')
train_df = train_df.query("周销量>0")
train_df.dropna(how='any', inplace=True)
train_df['商店ID'] = train_df['商店ID'].astype('float64')
train_df['年'] = train_df['年'].astype('float64')
train_df['周'] = train_df['周'].astype('float64')
train_df['节日A'] = train_df['节日A'].astype('float64')
train_df['节日B'] = train_df['节日B'].astype('float64')
train_df['节日C'] = train_df['节日C'].astype('float64')
train_df['商店模式'] = LabelEncoder() .fit_transform(train_df['商店模式'] ) #编码
train_df['商店级别'] = LabelEncoder() .fit_transform(train_df['商店级别'] ) #编码
train_df = train_df[["商店ID", "年",  "周",  "营业天数",  "打折天数",  "非节日",  "节日A",  "节日B",  "节日C",  "商店模式",  "商店级别",  "竞争者最近距离",  "周销量"]]
X, y =  train_df.drop(['周销量'], axis=1), train_df.周销量 #特征和目标
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3, shuffle=True, random_state=0) #划分训练测试集
print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)X_train, X_test = X_train.values, X_test.values
y_train, y_test = y_train.values.reshape(-1, 1), y_test.values.reshape(-1, 1)
X_train  = torch.from_numpy(X_train).type(torch.FloatTensor)
X_test = torch.from_numpy(X_test).type(torch.FloatTensor)
y_train = torch.from_numpy(y_train).type(torch.FloatTensor)
y_test = torch.from_numpy(y_test).type(torch.FloatTensor)
training_data = TensorDataset(X_train, y_train)
test_data = TensorDataset(X_test, y_test)
train_dataloader = DataLoader(training_data, batch_size=64)
test_dataloader = DataLoader(test_data, batch_size=64)class Net(nn.Module):      def __init__(self, features):super(Net, self).__init__() self.h1 = nn.Linear(features, 30, bias= True)self.a1 = nn.ReLU()self.h2 = nn.Linear(30, 10)self.a2 = nn.ReLU()self.regression = nn.Linear(10, 1)def forward(self, x): x = self.h1(x)x = self.a1(x)x = self.h2(x)x = self.a2(x)y_pred = self.regression(x)return y_predclass CustomLoss(nn.Module): #自定义损失函数def __init__(self):super(CustomLoss, self).__init__()def forward(self, x, y):loss = torch.sqrt(torch.sum(torch.pow((x-y)/x, 2))/len(x))return lossmodel = Net(features=X_train.shape[1])
print(model)model = Net(features=X_train.shape[1])
criterion = CustomLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)train_loss = []
test_loss = []epochs = 101
for epoch in range(epochs):model.train()for xb, yb in train_dataloader:pred = model(xb)loss = criterion(pred, yb)# loss.requires_grad_(True) optimizer.zero_grad()loss.backward()optimizer.step()if epoch%10==0:model.eval()with torch.no_grad():train_epoch_loss = sum(criterion(model(xb), yb) for xb, yb in train_dataloader)test_epoch_loss = sum(criterion(model(xb), yb) for xb, yb in test_dataloader)train_loss.append(train_epoch_loss.data.item() / len(train_dataloader))test_loss.append(test_epoch_loss.data.item() / len(test_dataloader))template = ("epoch:{:2d}, 训练损失:{:.5f}, 验证损失:{:.5f}")print(template.format(epoch, train_epoch_loss.data.item() / len(train_dataloader), test_epoch_loss.data.item() / len(test_dataloader)))
print('训练完成')fig = plt.figure(figsize = (6,4))
plt.plot(range(len(train_loss)), train_loss, label = "train_loss")
plt.plot(range(len(test_loss)), test_loss, label = "test_loss")
plt.legend()
plt.show()torch.save(model.state_dict(), ".\model_parameter.pth") #保存模型parameter = torch.load(".\model_parameter.pth") #加载模型
model.load_state_dict(parameter)

讨论

对于结构化的数据，深度学习不一定就比传统机器学习强，还需具体问题具体分析，不可一概而论。

参考文献

1，https://pytorch.org/tutorials/beginner/basics/quickstart_tutorial.html
2，https://shenzl.blog.csdn.net/article/details/128632831
3，https://sanhangkc.blog.csdn.net/article/details/128818696