ARM64架构手动编译libtorch，安装MKL/oneDNN加速模型推理，详细流程！

目录前言一、依赖环境二、下载pytorch源码三、下载oneDNN源码三、编译libtorch四、整理libtorch五、C调用libtorch前言libtorch官方并没有给出ARM64架构的安装文件在ARM64环境下libtorch需要手动编译。编译完成后发现模型推理速度太慢部分原因为手动编译的libtorch没有使用MKL加速工具安装MKL后重新编译libtorch后模型的推理速度提高了4-5倍。一、依赖环境cmake和gcc版本不能过低不然编译过程会报错。 gcc版本至少大于10.0。python环境所需安装pip install pyyaml二、下载pytorch源码git clone https://github.com/pytorch/pytorch --recursive cd pytorch # 下载子模块 git submodule sync git submodule update --init --recursive三、下载oneDNN源码cd到你的工作目录然后进行下载oneDNN源码文件。以我的工作目录为例cd /home/user/workspace wget https://github.com/oneapi-src/oneDNN/archive/refs/tags/v3.3.3.tar.gz --no-check-certificate安装配置# 解压 tar -xzvf v3.3.3.tar.gz cd oneDNN-3.3.3 # ARM平台代码适配 sed 2441s/.*/$(sed -n 2439p tests/gtests/graph/unit/backend/dnnl/test_conv.cpp)/ -i tests/gtests/graph/unit/backend/dnnl/test_conv.cpp # 创建构造目录 mkdir build cd build mkdir install # 编译注意修改路径 cmake -DCMAKE_INSTALL_PREFIX/home/user/workspace/oneDNN-3.3.3/build/install/ .. make -j make install # 检查安装目录是否有libdnnl.so 文件 ll install/lib64/三、编译libtorch修改环境变量# 华为服务器没有cuda关闭cuda export USE_CUDAFalse export BUILD_TESTFalse # 启用oneDNN加速库如果没有编译上述onnDNN源码则不添加该环境 export USE_MKLDNN1编译cd pytorch mkdir build cd build python ../tools/build_libtorch.py四、整理libtorch新建lintorch文件夹并将编译好的包整理到libtorch下。我的libtorch和pytorch在同一层级目录下。cd /home/user/workspace mkdir libtorch mkdir libtorch/share cp -r pytorch/torch/include/ libtorch/ cp -r pytorch/build/build/lib/ libtorch/ cp -r pytorch/torch/share/cmake/ libtorch/share/保证Libtorch的结构如下即可五、C调用libtorchCmakeList.txt文件示例cmake_minimum_required(VERSION 3.0 FATAL_ERROR) project(LibTorchDemo) set(CMAKE_CXX_STANDARD 17) set(CMAKE_CXX_STANDARD_REQUIRED True) # 设置 CMake 配置路径指向 libtorch 的 cmake 文件夹 set(CMAKE_PREFIX_PATH /home/user/workspace/libtorch/share/cmake) link_directories(/home/user/workspace/libtorch/lib) find_package(Torch REQUIRED) add_executable(digit digit.cpp) target_link_libraries(digit ${TORCH_LIBRARIES})六、进一步提高推理速度可选如果觉得速度还不够快可以开启BLAS加速。我的测试结果是可以在oneDNN加速后的推理速度上再提高2-2.5倍。安装openBLASsudo apt install libopenblas-dev gfortran进入pytorch目录并清理buildcd pytorch/build rm -rf * # 重新编译 python ../tools/build_libtorch.py重复步骤四整理libtorch文件。