MLU上使用MagicMind GFPGANv1.4 onnx加速！

---

前言

MagicMind是面向寒武纪MLU的推理加速引擎。MagicMind能将人工智能框架（TensorFlow、PyTorch、Caffe与ONNX等）训练好的算法模型转换成MagicMind统一计算图表示，并提供端到端的模型优化、代码生成以及推理业务部署能力。MagicMind 致力于为用户提供高性能、灵活、易用的编程接口以及配套工具，让用户能够专注于推理业务开发和部署本身，而无需过多关注底层硬件细节。
如果有用MLU、GPU、CPU训练好的算法模型，可以使用MagicMind快速地实现在MLU上部署推理业务。MagicMind的优势在于它能为MLU上的推理业务提供：

极致的性能优化。可靠的精度。尽可能少的内存占用。灵活的定制化开发能力。简洁易用的接口。

MagicMind适用（但不限于）以下推理业务场景：

图像处理（分类、检测、分割）。视频处理。自然语言处理。姿态检测。搜索、推荐。

MagicMind支持不同的系统平台和MLU硬件平台。MagicMind面向云端业务和端侧业务，提供了统一的编程界面，并针对两种业务场景的差异点，提供了必要的定制化功能（比如面向端侧部署提供了remote debug功能）。

具体参考：https://www.cambricon.com/docs/sdk_1.15.0/magicmind_1.7.0/user_guide/2_introduction/0_what_is_magicmind/what_is_magicmind.html

---

一、平台环境准备

镜像选择：pytorch:v24.10-torch2.4.0-torchmlu1.23.1-ubuntu22.04-py310 【本次mm操作，对镜像需求不是很高，只需对其相关版本即可】
卡选择：任意一款MLU3系列及以上卡

二、环境准备

1.GFPGAN代码处理

git clone https://github.com/xuanandsix/GFPGAN-onnxruntime-demo.git
#下载gfpgan原始模型
wget https://githubfast.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.4.pth
#转onnx操作
python torch2onnx.py  --src_model_path ./GFPGANv1.4.pth --dst_model_path ./GFPGANv1.4.onnx --img_size 512 
#onnx推理
python demo_onnx.py --model_path GFPGANv1.4.onnx --image_path ./cropped_faces/Adele_crop.png --save_path Adele_v3.jpg

性能：

(pytorch) root@notebook-mm-100semv-notebook-0:/workspace/volume/guojunceshi2/mmgfpgan/GFPGAN-onnxruntime-demo# python demo_onnx.py 
infer time: 2.8468078281730413
infer time: 2.2596635334193707
infer time: 3.117730548605323

2.MagicMind转换

#mmwhl包安装
pip install magicmind-1.13.0-cp310-cp310-linux_x86_64.whl
#代码拷贝
git clone https://gitee.com/cambricon/magicmind_cloud.git
#1,环境变量配置
cd magicmind_cloud/buildin/cv/classification/resnet50_onnx/

修改env.sh

export NEUWARE_HOME=/usr/local/neuware  #主要是这一行重要其余不变
export MM_RUN_PATH=${NEUWARE_HOME}/bin
#本sample工作路径
export PROJ_ROOT_PATH=$(cd $(dirname "${BASH_SOURCE[0]}");pwd)
export MAGICMIND_CLOUD=${PROJ_ROOT_PATH%buildin*}
export MODEL_PATH=${PROJ_ROOT_PATH}/data/models# CV类网络通用文件路径
export UTILS_PATH=${MAGICMIND_CLOUD}/buildin/cv/utils# Python公共组件路径
export PYTHON_COMMON_PATH=${MAGICMIND_CLOUD}/buildin/python_common
# CPP公共接口路径
export CPP_COMMON_PATH=$MAGICMIND_CLOUD/buildin/cpp_commonhas_add_common_path=$(echo ${PYTHONPATH}|grep "${PYTHON_COMMON_PATH}")
if [ -z ${has_add_common_path} ];thenexport PYTHONPATH=${PYTHONPATH}:${PYTHON_COMMON_PATH}
fihas_add_util_path=$(echo ${PYTHONPATH}|grep "${UTILS_PATH}")
if [ -z ${has_add_util_path} ];thenexport PYTHONPATH=${PYTHONPATH}:${UTILS_PATH}
fi

然后source env.sh

修改run.sh

#!/bin/bash
set -e
set -xmagicmind_model=face_force_float32_true
precision=force_float32
batch_size=1
dynamic_shape=falsepython gen_model.py --precision ${precision} \--input_dims ${batch_size} 3 512 512 \--batch_size ${batch_size} \-dynamic_shape ${dynamic_shape} \--magicmind_model ${magicmind_model} \--input_layout NHWC \--dim_range_min 1 3 512 512 \--dim_range_max 64 3 512 512 \--onnx /workspace/volume/guojunceshi2/mmgfpgan/GFPGAN-onnxruntime-demo/gfpgan14.onnx

参数解析

–precision 可选。精度模式，默认采用float32运行整个网络，即值为:force_float32。force_float32:所有算子以FLOAT32作为输入精度和输出数据类型,且中间结果也是FLOAT32。force float16:所有算子以FLOAT16作为输入精度和输出数据类型,且中间结果也是FLOAT16。qint8_mixed float32:模拟量化算子以FLOAT32作为输入，先量化成INT8，再转成FLOAT32进行计算，其他非量化算子的输入精度和输出数据类型和中间结果都是FLOAT32。qint16_mixed_foat32:模拟量化算子以FLOAT32作为输入,先量化成INT16,再转成FLOAT32进行计算，其他非量化算子的输入精度和输出数据类型和中间结果都是FLOAT32。
qint8_mixed _float16:模拟量化算子以FLOAT16作为输入，先量化成INT8，再转成FLOAT16进行计算，其他非量化算子的输入精度和输出数据类型和中间结果都是FLOAT16。ONNX支持的模拟量化算子包括:Conv1D，Conv2D,Conv3D,ConvTranspose1D，ConvTrans-pose2D，Gemm，MatMul。模拟量化相关概念见模拟量化。

–input_dims 输入维度
–dynamic_shape

运行

生成 face_force_float32_true文件

注意输入维度和输出维度

3.修改后模型运行

原始模型读取部分

img = img.transpose(0, 3, 1, 2)
pre_process 返回为1，3，512，512注意img输入维度为1，3，512，512
ort_inputs = {self.ort_session.get_inputs()[0].name: img}
ort_outs = self.ort_session.run(None, ort_inputs)

修改后

img = img.transpose(0, 1, 2, 3)
pre_process 返回为1，512，512，3
模型读取部分修改为，其余不变
记得推理前执行以下前面source env.sh操作
from mm_runner import MMRunner
self.ort_session = MMRunner(mm_file = "face_force_float32_true",device_id = 0)
ort_outs = self.ort_session([img])

运行效果

2025-01-06 10:49:16,886: INFO: mm_runner.py:20] Model instance Created Success!
2025-01-06 10:49:16,898: INFO: mm_runner.py:32] Model dev Created Success!
2025-01-06 10:49:17,516: INFO: mm_runner.py:39] Model engine Created Success!
2025-01-06 10:49:17,644: INFO: mm_runner.py:43] Model context Created Success!
2025-01-06 10:49:17,645: INFO: mm_runner.py:47] Model queue Created Success!
2025-01-06 10:49:17,645: INFO: mm_runner.py:50] Model inputs Created Success!
2025-01-06 10:49:17,645: INFO: mm_runner.py:51] All Model resource Created Success!
infer time: 0.11474167183041573
infer time: 0.04283882491290569
infer time: 0.040602266788482666
infer time: 0.04028203524649143
infer time: 0.04049760662019253
infer time: 0.04016706347465515
infer time: 0.04045788757503033
infer time: 0.04026786610484123
infer time: 0.041572125628590584
infer time: 0.04047401808202267
infer time: 0.04045314900577068
infer time: 0.04047247767448425
infer time: 0.04037348926067352
infer time: 0.04047695733606815
infer time: 0.04112406447529793

显存消耗

Every 2.0s: cnmon                                                                                              notebook-mm-100semv-notebook-0: Mon Jan  6 10:49:26 2025Mon Jan  6 10:49:26 2025
+------------------------------------------------------------------------------+
| CNMON v5.10.29                                               Driver v5.10.29 |
+-------------------------------+----------------------+-----------------------+
| Card  VF  Name       Firmware |               Bus-Id | Util        Ecc-Error |
| Fan   Temp      Pwr:Usage/Cap |         Memory-Usage | Mode     Compute-Mode |
|===============================+======================+=======================|
| 0     /   MLU370-M8    v1.1.4 |         0000:69:00.0 | 73%                 0 |
|  0%   34C        179 W/ 300 W |   731 MiB/ 42396 MiB | FULL          Default |
+-------------------------------+----------------------+-----------------------+
| 1     /   MLU370-M8    v1.1.4 |         0000:72:00.0 | 0%                  0 |
|  0%   27C         50 W/ 300 W |     0 MiB/ 42396 MiB | FULL          Default |
+-------------------------------+----------------------+-----------------------++------------------------------------------------------------------------------+
| Processes:                                                                   |
|  Card  MI  PID     Command Line                             MLU Memory Usage |
|==============================================================================|
|  0     /   40007   python                                            650 MiB |
+------------------------------------------------------------------------------+

优化前：2.84-3.0s

优化后：0.04-0.1s