快速体验fastllm安装部署并支持AMD ROCm推理加速
序言
fastllm是纯c++实现，无第三方依赖的高性能大模型推理库。
本文以国产海光DCU为例，在AMD ROCm平台下编译部署fastllm以实现LLMs模型推理加速。
测试平台：曙光超算互联网平台SCNet
GPU/DCU：异构加速卡AI 显存64GB PCIE（基于ROCm平台的GPU）
服务器的详细配置，请参考：曙光超算互联网平台SCNet之国产异构加速卡DCU
一、参考资料
fastllm 代码仓库：https://github.com/lcpu-club/fastllm-rocm
二、快速开始
1. fastllm功能概述
- 🚀 纯c++实现，便于跨平台移植，可以在安卓上直接编译。
- 🚀 ARM平台支持NEON指令集加速，X86平台支持AVX指令集加速，NVIDIA平台支持CUDA加速。
- 🚀 支持浮点模型（FP32), 半精度模型(FP16), 量化模型(INT8, INT4) 加速。
- 🚀 支持多卡部署，支持GPU + CPU混合部署。
- 🚀 支持Batch速度优化。
- 🚀 支持并发计算时动态拼Batch。
- 🚀 支持流式输出，很方便实现打字机效果。
- 🚀 支持python调用。
- 🚀 前后端分离设计，便于支持新的计算设备。
- 🚀 目前支持ChatGLM模型，各种LLAMA模型(ALPACA, VICUNA等)，BAICHUAN模型，MOSS模型。
2. 测试环境
root@notebook-1823641624653922306-scnlbe5oi5-42808:/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/build# hipconfig
HIP version : 5.7.24164== hipconfig
HIP_PATH : /opt/dtk/hip
ROCM_PATH : /opt/dtk
HIP_COMPILER : clang
HIP_PLATFORM : amd
HIP_RUNTIME : rocclr
CPP_CONFIG : -D__HIP_PLATFORM_HCC__= -D__HIP_PLATFORM_AMD__= -I"/opt/dtk/hip/include" -I"/opt/dtk/llvm/bin/../lib/clang/15.0.0" -I/opt/dtk/hsa/include== hip-clang
HSA_PATH : /opt/dtk/hsa
HIP_CLANG_PATH : /opt/dtk/llvm/bin
clang version 15.0.0
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/dtk/llvm/bin
AOMP-15.0-61 (http://github.com/ROCm-Developer-Tools/aomp):Source ID:15.0-61-595b0d8133fafef5742f7d39f8e6a07b31afff56LLVM version 15.0.0gitOptimized build.Default target: x86_64-unknown-linux-gnuHost CPU: (unknown)Registered Targets:amdgcn - AMD GCN GPUsr600 - AMD GPUs HD2XXX-HD6XXXx86 - 32-bit X86: Pentium-Pro and abovex86-64 - 64-bit X86: EM64T and AMD64
hip-clang-cxxflags : -std=c++11 -isystem /opt/dtk-24.04/llvm/lib/clang/15.0.0/include/.. -isystem /opt/dtk-24.04/llvm/lib/clang/15.0.0/include/cuda_wrappers -isystem /opt/dtk/hsa/include -isystem /opt/dtk/hip/include -msram-ecc -msram-ecc -msram-ecc -O3
hip-clang-ldflags : --driver-mode=g++ -L"/opt/dtk/hip/lib" -L"/opt/dtk/lib" -O3 -lgcc_s -lgcc -lpthread -lm -lrt=== Environment Variables
PATH=/usr/local/bin:/opt/mpi/bin:/opt/hwloc/bin/:/opt/cmake/bin/:/opt/conda/bin:/opt/conda/bin:/opt/dtk/bin:/opt/dtk/llvm/bin:/opt/dtk/hip/bin:/opt/dtk/hip/bin/hipify:/opt/hyhal/bin:/opt/mpi/bin:/opt/hwloc/bin/:/opt/cmake/bin/:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/conda/bin
LD_LIBRARY_PATH=/usr/local/lib/:/usr/local/lib64/:/opt/mpi/lib:/opt/hwloc/lib:/opt/dtk/hip/lib:/opt/dtk/llvm/lib:/opt/dtk/lib:/opt/dtk/lib64:/opt/hyhal/lib:/opt/hyhal/lib64:/opt/dtk-24.04/hip/lib:/opt/dtk-24.04/llvm/lib:/opt/dtk-24.04/lib:/opt/dtk-24.04/lib64:/opt/hyhal/lib:/opt/hyhal/lib64:/usr/local/lib/:/usr/local/lib64/:/opt/mpi/lib:/opt/hwloc/lib:
HIP_PATH=/opt/dtk/hip== Linux Kernel
Hostname : notebook-1823641624653922306-scnlbe5oi5-42808
Linux notebook-1823641624653922306-scnlbe5oi5-42808 3.10.0-957.el7.x86_64 #1 SMP Fri Oct 25 22:34:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
3. 安装CLBlast
CLBlast: Building and installing
4. cmake编译
# 下载源码
git clone https://github.com/lcpu-club/fastllm-rocm.gitcd fastllm
mkdir build
cd build# 设置环境变量
export ROCM_PATH="/opt/dtk"
export CLBlast_DIR="/usr/lib/x86_64-linux-gnu/cmake/CLBlast"export CXXFLAGS="-I${ROCM_PATH}/include -Wno-pass-failed -D__HIP_PLATFORM_AMD__"# cmake
cmake .. -DUSE_ROCM=ON -DCMAKE_CXX_COMPILER=/opt/dtk/llvm/bin/clang++
root@notebook-1823641624653922306-scnlbe5oi5-42808:/public/home/scnlbe5oi5/Downloads/cache/fastl
lm-rocm/build# cmake .. -DUSE_ROCM=ON -DCMAKE_CXX_COMPILER=/opt/dtk/llvm/bin/clang++
-- The CXX compiler identification is Clang 15.0.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/dtk/llvm/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- USE_CUDA: OFF
-- USE_ROCM: ON
-- PYTHON_API: OFF
-- USE_SENTENCEPIECE: OFF
-- CMAKE_CXX_FLAGS-I/opt/dtk/include -Wno-pass-failed -D__HIP_PLATFORM_AMD__ -pthread --std=c++17 -O3 -march=native
-- The HIP compiler identification is Clang 15.0.0
-- Detecting HIP compiler ABI info
-- Detecting HIP compiler ABI info - done
-- Check for working HIP compiler: /opt/dtk/llvm/bin/clang++ - skipped
-- Detecting HIP compile features
-- Detecting HIP compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS
-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS - Success
-- System architecture is x86_64
-- Configuring done (5.1s)
-- Generating done (0.6s)
-- Build files have been written to: /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/build
5. make编译
单线程编译
root@notebook-1823641624653922306-scnlbe5oi5-42808:/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/bu
ild# make -j1
[ 2%] Building CXX object CMakeFiles/fastllm.dir/src/fastllm.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:46: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:32: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:18: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
9 warnings generated.
[ 4%] Building CXX object CMakeFiles/fastllm.dir/src/device.cpp.o
[ 6%] Building CXX object CMakeFiles/fastllm.dir/src/model.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:3:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:100:16: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested herereturn std::unique_ptr<fastllm::basellm> (model);^
1 warning generated.
[ 8%] Building CXX object CMakeFiles/fastllm.dir/src/executor.cpp.o
[ 11%] Building CXX object CMakeFiles/fastllm.dir/src/devices/cpu/cpudevice.cpp.o
[ 13%] Building CXX object CMakeFiles/fastllm.dir/src/devices/cpu/cpudevicebatch.cpp.o
[ 15%] Building CXX object CMakeFiles/fastllm.dir/src/models/chatglm.cpp.o
[ 17%] Building CXX object CMakeFiles/fastllm.dir/src/models/moss.cpp.o
[ 20%] Building CXX object CMakeFiles/fastllm.dir/src/models/llama.cpp.o
[ 22%] Building CXX object CMakeFiles/fastllm.dir/src/models/qwen.cpp.o
[ 24%] Building CXX object CMakeFiles/fastllm.dir/src/models/basellm.cpp.o
[ 26%] Building CXX object CMakeFiles/fastllm.dir/src/models/glm.cpp.o
[ 28%] Building CXX object CMakeFiles/fastllm.dir/src/devices/rocm/rocmdevice.cpp.o
[ 31%] Building CXX object CMakeFiles/fastllm.dir/src/devices/rocm/rocmdevicebatch.cpp.o
[ 33%] Building HIP object CMakeFiles/fastllm.dir/src/devices/rocm/fastllm-rocm.hip.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
4 warnings generated when compiling for gfx906.
4 warnings generated when compiling for gfx926.
4 warnings generated when compiling for gfx928.
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
4 warnings generated when compiling for host.
[ 33%] Built target fastllm
[ 35%] Building CXX object CMakeFiles/main.dir/main.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/main.cpp:1:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/main.cpp:64:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
1 warning generated.
[ 37%] Linking HIP executable main
[ 37%] Built target main
[ 40%] Building CXX object CMakeFiles/quant.dir/tools/src/quant.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/quant.cpp:6:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/quant.cpp:49:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
1 warning generated.
[ 42%] Linking HIP executable quant
[ 42%] Built target quant
[ 44%] Building CXX object CMakeFiles/testOps.dir/test/ops/cppOps.cpp.o
[ 46%] Linking HIP executable testOps
[ 46%] Built target testOps
[ 48%] Building CXX object CMakeFiles/webui.dir/example/webui/webui.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/webui/webui.cpp:4:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/webui/webui.cpp:76:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
1 warning generated.
[ 51%] Linking HIP executable webui
[ 51%] Built target webui
[ 53%] Building CXX object CMakeFiles/benchmark.dir/example/benchmark/benchmark.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/benchmark/benchmark.cpp:5:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/benchmark/benchmark.cpp:82:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
1 warning generated.
[ 55%] Linking HIP executable benchmark
[ 55%] Built target benchmark
[ 57%] Building CXX object CMakeFiles/apiserver.dir/example/apiserver/apiserver.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/apiserver.cpp:4:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/apiserver.cpp:231:8: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested here
struct WorkQueue {^
1 warning generated.
[ 60%] Building CXX object CMakeFiles/apiserver.dir/example/apiserver/json11.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:251:58: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonString(string &&value) : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:259:62: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonArray(Json::array &&value) : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:267:64: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonObject(Json::object &&value) : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:309:76: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(string &&value) : m_ptr(make_shared<JsonString>(move(value))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:312:75: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(Json::array &&values) : m_ptr(make_shared<JsonArray>(move(values))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:314:76: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(Json::object &&values) : m_ptr(make_shared<JsonObject>(move(values))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:412:29: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]return fail(move(msg), Json());^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:251:52: note: in instantiation of member function 'json11::Value<json11::Json::STRING, std::basic_string<char>>::Value' requested hereexplicit JsonString(string &&value) : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:259:56: note: in instantiation of member function 'json11::Value<json11::Json::ARRAY, std::vector<json11::Json>>::Value' requested hereexplicit JsonArray(Json::array &&value) : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:267:58: note: in instantiation of member function 'json11::Value<json11::Json::OBJECT, std::map<std::basic_string<char>, json11::Json>>::Value' requested hereexplicit JsonObject(Json::object &&value) : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:272:22: note: in instantiation of member function 'json11::Value<json11::Json::NUL, json11::NullStruct>::Value' requested hereJsonNull() : Value({}) {}^
11 warnings generated.
[ 62%] Linking HIP executable apiserver
[ 62%] Built target apiserver
[ 64%] Building CXX object CMakeFiles/fastllm_tools.dir/src/fastllm.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:46: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:32: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:18: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
9 warnings generated.
[ 66%] Building CXX object CMakeFiles/fastllm_tools.dir/src/device.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/model.cpp.o
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:3:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:100:16: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested herereturn std::unique_ptr<fastllm::basellm> (model);^
1 warning generated.
[ 71%] Building CXX object CMakeFiles/fastllm_tools.dir/src/executor.cpp.o
[ 73%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/cpu/cpudevice.cpp.o
[ 75%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/cpu/cpudevicebatch.cpp.o
[ 77%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/chatglm.cpp.o
[ 80%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/moss.cpp.o
[ 82%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/llama.cpp.o
[ 84%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/qwen.cpp.o
[ 86%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/basellm.cpp.o
[ 88%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/glm.cpp.o
[ 91%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/rocm/rocmdevice.cpp.o
[ 93%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/rocm/rocmdevicebatch.cpp.o
[ 95%] Building HIP object CMakeFiles/fastllm_tools.dir/src/devices/rocm/fastllm-rocm.hip.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9:/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip ^~~~~~~~~~~
:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
4 warnings generated when compiling for gfx926.
4 warnings generated when compiling for gfx928.
4 warnings generated when compiling for gfx906.
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
4 warnings generated when compiling for host.
[ 97%] Building CXX object CMakeFiles/fastllm_tools.dir/tools/src/pytools.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:78:42: warning: 'make_config' has C-linkage specified, but returns user-defined type 'fastllm::GenerationConfig' which is incompatible with C [-Wreturn-type-c-linkage]DLL_EXPORT fastllm::GenerationConfig make_config(int max_length, bool do_sample, float top_p, int top_k,^
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:5:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:95:29: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested heremodels.models[id] = fastllm::CreateLLMModelFromFile(path);^
2 warnings generated.
[100%] Linking HIP shared library libfastllm_tools.so
[100%] Built target fastllm_tools
多线程编译
root@notebook-1823641624653922306-scnlbe5oi5-42808:/public/home/scnlbe5oi5/Downloads/cache/fastl
lm-rocm/build# make -j${nproc}
[ 6%] Building CXX object CMakeFiles/fastllm.dir/src/fastllm.cpp.o
[ 17%] Building CXX object CMakeFiles/fastllm.dir/src/device.cpp.o
[ 6%] Building CXX object CMakeFiles/fastllm.dir/src/model.cpp.o
[ 46%] Building CXX object CMakeFiles/fastllm.dir/src/executor.cpp.o
[ 46%] Building CXX object CMakeFiles/fastllm_tools.dir/src/fastllm.cpp.o
[ 48%] Building CXX object CMakeFiles/fastllm.dir/src/devices/cpu/cpudevice.cpp.o
[ 53%] Building CXX object CMakeFiles/fastllm.dir/src/models/chatglm.cpp.o
[ 55%] Building CXX object CMakeFiles/fastllm_tools.dir/src/device.cpp.o
[ 66%] Building CXX object CMakeFiles/fastllm.dir/src/devices/cpu/cpudevicebatch.cpp.o
[ 66%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/rocm/rocmdevicebatch.cpp.o
[ 68%] Building HIP object CMakeFiles/fastllm_tools.dir/src/devices/rocm/fastllm-rocm.hip.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/model.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/models/moss.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/cpu/cpudevice.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/models/basellm.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/executor.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/cpu/cpudevicebatch.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/models/llama.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/models/glm.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/models/qwen.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/chatglm.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/moss.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/qwen.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/llama.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/devices/rocm/rocmdevice.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm.dir/src/devices/rocm/rocmdevicebatch.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/devices/rocm/rocmdevice.cpp.o
[ 68%] Building HIP object CMakeFiles/fastllm.dir/src/devices/rocm/fastllm-rocm.hip.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/glm.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/src/models/basellm.cpp.o
[ 68%] Building CXX object CMakeFiles/fastllm_tools.dir/tools/src/pytools.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:78:42: warning: 'make_config' has C-linkage specified, but returns user-defined type 'fastllm::GenerationConfig' which is incompatible with C [-Wreturn-type-c-linkage]DLL_EXPORT fastllm::GenerationConfig make_config(int max_length, bool do_sample, float top_p, int top_k,^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:5:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/pytools.cpp:95:29: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested heremodels.models[id] = fastllm::CreateLLMModelFromFile(path);^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:46: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:32: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:18: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:908:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:50: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:36: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1014:22: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:46: warning: implicit conversion from 'int' to 'char' changes value from 129 to -127 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:32: warning: implicit conversion from 'int' to 'char' changes value from 150 to -106 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:1286:18: warning: implicit conversion from 'int' to 'char' changes value from 226 to -30 [-Wconstant-conversion]blank += 226, blank += 150, blank += 129;~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:3:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:100:16: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested herereturn std::unique_ptr<fastllm::basellm> (model);^
2 warnings generated.
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:3:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:100:16: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested herereturn std::unique_ptr<fastllm::basellm> (model);^
1 warning generated.
4 warnings generated when compiling for gfx928.
1 warning generated.
4 warnings generated when compiling for gfx926.
4 warnings generated when compiling for gfx906.
4 warnings generated when compiling for gfx906.
4 warnings generated when compiling for gfx926.
4 warnings generated when compiling for gfx928.
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:30:17: warning: 4 enumeration values not handled in switch: 'HIPBLAS_STATUS_SUCCESS', 'HIPBLAS_STATUS_HANDLE_IS_NULLPTR', 'HIPBLAS_STATUS_INVALID_ENUM'... [-Wswitch]switch (code) {^~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:81:9: warning: comparison of different enumeration types ('hipError_t' and 'hipblasStatus_t') [-Wenum-compare]checkHipErrors(stat);^~~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:19:20: note: expanded from macro 'checkHipErrors'if (hipSuccess != (a)) { \~~~~~~~~~~ ^ ~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1518:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", ret);~~ ^~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:1571:32: warning: format specifies type 'unsigned int' but the argument has type 'void *' [-Wformat]DEBUG_PRINT("hipFree: %x", rocmBuffers[i].data);~~ ^~~~~~~~~~~~~~~~~~~
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/devices/rocm/fastllm-rocm.hip:16:53: note: expanded from macro 'DEBUG_PRINT'__LINE__, __func__, __VA_ARGS__); fprintf(stderr, "\n"); } while (0)^~~~~~~~~~~
4 warnings generated when compiling for host.
4 warnings generated when compiling for host.
9 warnings generated.
9 warnings generated.
[ 71%] Linking HIP shared library libfastllm_tools.so
[ 71%] Built target fastllm_tools
[ 71%] Built target fastllm
[ 77%] Building CXX object CMakeFiles/testOps.dir/test/ops/cppOps.cpp.o
[ 77%] Building CXX object CMakeFiles/benchmark.dir/example/benchmark/benchmark.cpp.o
[ 77%] Building CXX object CMakeFiles/main.dir/main.cpp.o
[ 86%] Building CXX object CMakeFiles/webui.dir/example/webui/webui.cpp.o
[ 86%] Building CXX object CMakeFiles/quant.dir/tools/src/quant.cpp.o
[ 86%] Building CXX object CMakeFiles/apiserver.dir/example/apiserver/apiserver.cpp.o
[ 86%] Building CXX object CMakeFiles/apiserver.dir/example/apiserver/json11.cpp.o
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:251:58: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonString(string &&value) : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:259:62: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonArray(Json::array &&value) : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:267:64: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit JsonObject(Json::object &&value) : Value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:309:76: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(string &&value) : m_ptr(make_shared<JsonString>(move(value))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:312:75: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(Json::array &&values) : m_ptr(make_shared<JsonArray>(move(values))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:314:76: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]Json::Json(Json::object &&values) : m_ptr(make_shared<JsonObject>(move(values))) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:412:29: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]return fail(move(msg), Json());^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:251:52: note: in instantiation of member function 'json11::Value<json11::Json::STRING, std::basic_string<char>>::Value' requested hereexplicit JsonString(string &&value) : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:259:56: note: in instantiation of member function 'json11::Value<json11::Json::ARRAY, std::vector<json11::Json>>::Value' requested hereexplicit JsonArray(Json::array &&value) : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:267:58: note: in instantiation of member function 'json11::Value<json11::Json::OBJECT, std::map<std::basic_string<char>, json11::Json>>::Value' requested hereexplicit JsonObject(Json::object &&value) : Value(move(value)) {}^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:204:50: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]explicit Value(T &&value) : m_value(move(value)) {}^std::
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/json11.cpp:272:22: note: in instantiation of member function 'json11::Value<json11::Json::NUL, json11::NullStruct>::Value' requested hereJsonNull() : Value({}) {}^
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/quant.cpp:6:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/tools/src/quant.cpp:49:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/main.cpp:1:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/main.cpp:64:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
[ 88%] Linking HIP executable testOps
1 warning generated.
[ 91%] Linking HIP executable quant
[ 91%] Built target testOps
1 warning generated.
[ 93%] Linking HIP executable main
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/benchmark/benchmark.cpp:5:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/benchmark/benchmark.cpp:82:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
[ 93%] Built target quant
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/apiserver.cpp:4:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/apiserver/apiserver.cpp:231:8: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested here
struct WorkQueue {^
[ 93%] Built target main
1 warning generated.
[ 95%] Linking HIP executable benchmark
[ 95%] Built target benchmark
11 warnings generated.
1 warning generated.
[ 97%] Linking HIP executable apiserver
[ 97%] Built target apiserver
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/webui/webui.cpp:4:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/example/webui/webui.cpp:76:18: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested hereauto model = fastllm::CreateLLMModelFromFile(config.path);^
1 warning generated.
[100%] Linking HIP executable webui
[100%] Built target webui
6. 运行demo
准备模型，以 huangyuyang/chatglm2-6b-int4.flm 为例。
# 命令行聊天程序
./main -p chatglm2-6b-int4.flm
root@notebook-1823641624653922306-scnlbe5oi5-42808:/public/home/scnlbe5oi5/Downloads/cache/fastl
ild# ./main -p chatglm2-6b-int4.flm
AVX: ON
AVX2: ON
AARCH64: OFF
Neon FP16: OFF
Neon DOT: OFF
Load (200 / 200)
Warmup...
Launch params (512, 1, 1) are larger than launch bounds (256) for kernel _Z26FastllmRMSNormKerne512EEvPfS0_S0_iif please add __launch_bounds__ to kernel define or use --gpu-max-threads-per-blole program !
finish.
欢迎使用 chatglm 模型. 输入内容对话，reset清空历史记录，stop退出程序.
用户: hi
chatglm: Hello! How can I assist you today?
用户: 中国深圳有哪些旅游景点
chatglm: 深圳是一个美丽的城市,有很多值得一游的景点。以下是一些深圳的旅游景点:1. 深圳湾公园:是一座大型公园,位于深圳湾海滨,可以欣赏到美丽的海景和城市景观。2. 华侨城:这是一个综合性的文化城,里面有很多娱乐设施和景点,如大型主题公园、音乐厅和博物馆等。3. 深圳会展中心:这是一个现代化的展览中心,可以举办各种展览和活动。4. 东部华侨城:这是一个以海洋文化为主题的大型综合性景区,里面有很多海洋生物和各种娱乐项目。5. 深圳中心书城:这是一个大型的书店,里面有很多图书和文化产品。6. 深圳动物园:这是一个大型的动物园,可以了解到各种动物和它们的生活习性。7. 大梅沙海滨公园:这是一个美丽的海滨公园,里面有很多户外活动设施和景观。8. 深圳技术馆:这是一个展示科技和文化的博物馆,可以了解到各种科技发展的趋势和文化。9. 深圳音乐厅:这是一个现代化的音乐厅,可以举办各种音乐演出和活动。10. 深圳海底世界:这是一个大型的海洋生物博物馆,可以了解到各种海洋生物和它们的生活习性。
运行时的资源占用情况
三、FAQ
Q：#error ("Must define exactly one of __HIP_PLATFORM_AMD__ or __HIP_PLATFORM_NVIDIA__"
Must define exactly one of __HIP_PLATFORM_AMD__ or __HIP_PLATFORM_NVIDIA__ Error #1789
/opt/dtk/include/hip/hip_vector_types.h:38:2: error: #error ("Must define exactly one of __HIP_PLATFORM_AMD__ or __HIP_PLATFORM_NVIDIA__");38 | #error("Must define exactly one of __HIP_PLATFORM_AMD__ or __HIP_PLATFORM_NVIDIA__");| ^~~~~
In file included from /opt/dtk/include/hip/hip_runtime.h:113,from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/devices/rocm/fastllm-rocm.hiph:2,from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/fastllm.cpp:38:
/opt/dtk/include/hip/hip_runtime_api.h:8549:61: error: 'hipHostMallocDefault' was not declared in this scope8549 | unsigned int flags = hipHostMallocDefault) {| ^~~~~~~~~~~~~~~~~~~~
/opt/dtk/include/hip/hip_runtime_api.h:8568:61: error: 'hipMemAttachGlobal' was not declared in this scope8568 | unsigned int flags = hipMemAttachGlobal) {| ^~~~~~~~~~~~~~~~~~
make[2]: *** [CMakeFiles/fastllm.dir/build.make:76: CMakeFiles/fastllm.dir/src/fastllm.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:97: CMakeFiles/fastllm.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
错误原因：未指定 D__HIP_PLATFORM_AMD__。
解决方法：
# 设置环境变量
export CXXFLAGS="-I${ROCM_PATH}/include -Wno-pass-failed -D__HIP_PLATFORM_AMD__"
Q：error: use of undeclared identifier 'CUR_DEVICE' contextLayer.ToDevice(CUR_DEVICE)
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/models/chatglm.cpp:382:31: error: use of undeclared identifier 'CUR_DEVICE'hiddenStates.ToDevice(CUR_DEVICE);^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/models/chatglm.cpp:518:34: error: use of undeclared identifier 'CUR_DEVICE'pastKey.ToDevice(CUR_DEVICE);^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/models/chatglm.cpp:519:36: error: use of undeclared identifier 'CUR_DEVICE'pastValue.ToDevice(CUR_DEVICE);^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/models/chatglm.cpp:668:43: error: use of undeclared identifier 'CUR_DEVICE'contextLayer.ToDevice(CUR_DEVICE);^
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:3:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/model.h:8:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/models/basellm.h:2:
In file included from /public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/include/fastllm.h:19:
In file included from /usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/memory:80:
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:81:2: warning: delete called on 'fastllm::basellm' that is abstract but has non-virtual destructor [-Wdelete-abstract-non-virtual-dtor]delete __ptr;^
/usr/lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:292:4: note: in instantiation of member function 'std::default_delete<fastllm::basellm>::operator()' requested hereget_deleter()(std::move(__ptr));^
/public/home/scnlbe5oi5/Downloads/cache/fastllm-rocm/src/model.cpp:100:16: note: in instantiation of member function 'std::unique_ptr<fastllm::basellm>::~unique_ptr' requested herereturn std::unique_ptr<fastllm::basellm> (model);^
1 error generated.
make[2]: *** [CMakeFiles/fastllm.dir/build.make:90: CMakeFiles/fastllm.dir/src/device.cpp.o] Error 1
错误原因：未指定 DUSE_ROCM。
解决方法：
cmake .. -DUSE_ROCM=ON -DCMAKE_CXX_COMPILER=/opt/dtk/llvm/bin/clang++
相关文章：
快速体验fastllm安装部署并支持AMD ROCm推理加速
序言 fastllm是纯c实现，无第三方依赖的高性能大模型推理库。 本文以国产海光DCU为例，在AMD ROCm平台下编译部署fastllm以实现LLMs模型推理加速。 测试平台：曙光超算互联网平台SCNet GPU/DCU：异构加速卡AI 显存64GB PCIE&#…...
报错：java: javacTask: 源发行版 8 需要目标发行版 1.8
程序报错： Executing pre-compile tasks... Loading Ant configuration... Running Ant tasks... Running before tasks Checking sources Copying resources... [gulimail-coupon] Copying resources... [gulimail-common] Parsing java… [gulimail-common] java…...
【数据结构篇】~单链表（附源码）
【数据结构篇】~链表 链表前言链表的实现1.头文件2.源文件 链表前言 链表是一种物理存储结构上非连续、非顺序的存储结构，数据元素的逻辑顺序是通过链表中的指针链接次序实现的。 1、链式机构在逻辑上是连续的，在物理结构上不一定连续 2、结点一般是从…...
旋转图像（LeetCode）
题目 给定一个 n n 的二维矩阵 matrix 表示一个图像。请你将图像顺时针旋转 90 度。 你必须在 原地 旋转图像，这意味着你需要直接修改输入的二维矩阵。请不要 使用另一个矩阵来旋转图像。 解题 def rotate(matrix):n len(matrix)# 矩阵转置for i in range(n):for…...
入门 - vue中v-model的实现原理和完整用法详解
v-model介绍 v-model是vue的双向绑定的指令，能将页面上控件输入的值同步更新到相关绑定的data属性，也会在更新data绑定属性时候，更新页面上输入控件的值。在view层，model层相互需要数据交互，即可使用v-model。 双向绑…...
【区块链+金融服务】港融区域股权服务平台 | FISCO BCOS应用案例
中国证监会在 2020 年启动了区块链建设试点工作，提出建设基于区块链的场外市场登记系统和交易报告库，利 用区块链去中心化、不易篡改、安全稳定等技术特点，构建区域性股权市场数字化信任机制，为区域性股权市场 提供基础支撑设施。…...
Nginx反向代理和前后端分离项目打包部署
Nginx反向代理 Nginx的定位：主要用于做反向代理，一般都是用它来做前端页面的服务器，动态资源代理到后端服务器。这样做的好处是可以避免跨域请求带来的不便。 使用Nginx主要是对Nginx中的nginx.conf文件进行配置： 虚拟主机配置…...
Spring 中ApplicationContext
ApplicationContext 是 Spring 框架中最重要的接口之一，用于提供 Spring IoC 容器的功能。它是一个比 BeanFactory 更高级的容器，负责管理 Spring bean 的生命周期，同时提供对各种企业服务的集成，例如事件传播、国际化、弱引用等。…...
python之时间 datetime、date、time、timedelta、dateutil
在 Python 中，处理日期和时间的常用库是 datetime。此外，还有一些第三方库如 pytz 和 dateutil 可以帮助处理时区和日期解析。 1. 使用 datetime 模块 导入模块 from datetime import datetime, date, time, timedelta获取当前日期和时间 now datet…...
【机器学习第11章——特征选择与稀疏学习】
机器学习第11章——特征选择与稀疏学习 11.特征选择与稀疏学习11.1子集搜索与评价子集搜索子集评价 11.2 过滤式选择11.3 包裹式选择11.4 嵌入式选择11.5 稀疏表示与字典学习稀疏表示字典学习 11.6 压缩感知 11.特征选择与稀疏学习 11.1子集搜索与评价 特征：描述物…...
LeetCode-day43-3137. K 周期字符串需要的最少操作次数
LeetCode-day43-3137. K 周期字符串需要的最少操作次数 题目描述示例示例1：示例2： 思路代码 题目描述 给你一个长度为 n 的字符串 word 和一个整数 k ，其中 k 是 n 的因数。 在一次操作中，你可以选择任意两个下标 i 和 j&#x…...
基于springboot的智能家居系统
TOC springboot198基于springboot的智能家居系统 研究背景与现状 时代的进步使人们的生活实现了部分自动化，由最初的全手动办公已转向手动自动相结合的方式。比如各种办公系统、智能电子电器的出现，都为人们生活的享受提供帮助。采用新型的自动化方式…...
【从问题中去学习k8s】k8s中的常见面试题（夯实理论基础）（七）
本站以分享各种运维经验和运维所需要的技能为主 《python零基础入门》：python零基础入门学习 《python运维脚本》： python运维脚本实践 《shell》：shell学习 《terraform》持续更新中：terraform_Aws学习零基础入门到最佳实战 《k8…...
C:每日一练：单身狗（2.0版本）
前言： 今天在刷题的时候突然看到一道题，疑似一位故题。仔细一看，欸！这不是就是单身狗的升级版吗？我想那必须再安排一篇，不过由于本篇文章与上一篇单身狗文章所涉及的知识点基本相同，所以还请大…...
打破接口壁垒：适配器模式让系统无缝对接
适配器模式（Adapter Pattern）是一种结构型设计模式，它允许不兼容的接口之间协同工作。主要用途是将一个类的接口转换成客户期望的另一个接口，使得原本接口不兼容的对象可以一起工作。 一、适配器模式的组成 目标接口（…...
U-Boot 命令使用
U-Boot 是一种常用的引导加载程序，用于引导嵌入式系统。它提供了一系列命令以进行系统配置、引导操作和调试。 以下是一些常见的 U-Boot 命令及其用法： bootm：从指定的内存地址启动操作系统映像。 用法：bootm [addr] bootz&…...
谷歌的高级指令有哪些
今天会分享一些组合用法，这样就能节省许多时间可以放在跟进客户上面（本文只介绍谷歌的搜索指令，并无推广） part one 谷歌常用的搜索引擎指令： 1、Inurl，在网址中 2、Intext，在网页内容中 3、…...
Redis操作--RedisTemplate（一）介绍
一、介绍 1、简介 RedisTemplate 是 Spring Data Redis 提供的一个高级抽象，由 Spring 官方提供的方便操作 Redis 数据库的一个工具类，支持模板设计模式，使得操作 Redis 更加符合 Spring 的编程模型。还支持序列化机制，可以处理…...
GitLab环境搭建
GitLab环境搭建 一、环境搭建 1、更新系统软件包： sudo yum update2、安装docker sudo yum install -y yum-utils device-mapper-persistent-data lvm2 sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo sudo yum install do…...
Socket编程TCP 基础
一.什么是Socket(套接字） 定义：就是对网络中不同主机上的应用进程之间进行双向通信的端点的抽象。一个套接字就是网络上进程通信的一端，提供了应用层进程利用网络协议交换数据的机制。从所处的地位来讲，套接字上联应用进程&#x…...
JAVA中的Iterator与ListIterator
Java中的Iterator类是Java集合框架中的一个重要接口，它用于遍历集合中的元素。Iterator提供了三个基本操作：检查是否有下一个元素、获取下一个元素以及移除元素。下面将详细介绍Iterator类及其使用方法，并提供相应的代码例子和中文注释。 一、…...
高校疫情防控web系统pf
TOC springboot365高校疫情防控web系统pf 第1章 绪论 1.1 课题背景 互联网发展至今，无论是其理论还是技术都已经成熟，而且它广泛参与在社会中的方方面面。它让信息都可以通过网络传播，搭配信息管理工具可以很好地为人们提供服务。所以各行…...
复现nnUNet2并跑通自定义数据
复现nnUNet2并跑通自定义数据 1. 配置环境2. 处理数据集2.1 创建文件夹2.2 数据集格式转换2.3 数据集预处理 3. 训练4. 改进模型4.1 概要4.2 加注意力模块 1. 配置环境 stage1：创建python环境，这里建议python3.10 conda create --n nnunet python3.10 …...
Educational Codeforces Round 169 (Rated for Div. 2)(ABCDE)
A. Closest Point 签到 #define _rep(i,a,b) for(int i(a);i<(b);i) int n,m; int q[N]; void solve() {cin>>n;_rep(i,1,n)cin>>q[i];if(n!2)cout<<"NO\n";else if(abs(q[1]-q[2])!1)cout<<"YES\n";else cout<<"…...
成为Python砖家(2): str 最常用的8大方法
str 类最常用的8个方法 str.lower()str.upper()str.split(sepNone, maxsplit-1)str.count(sub[, start[, end]])str.replace(old, new[, count])str.center(width[, fillchar])str.strip([chars])str.join(iterable) 查询方法的文档 根据 成为Python砖家(1): 在本地查询Pyth…...
深入理解JVM运行时数据区（内存布局 ）5大部分 | 异常讨论
前言： JVM运行时数据区（内存布局）是Java程序执行时用于存储各种数据的内存区域。这些区域在JVM启动时被创建，并在JVM关闭时销毁。它们的布局和管理方式对Java程序的性能和稳定性有着重要影响。 目录 一、由以下5大部分组成 1.…...
JAVA根据表名获取Oracle表结构信息
响应实体封装 import lombok.AllArgsConstructor; import lombok.Builder; import lombok.Data; import lombok.NoArgsConstructor;/*** author CQY* version 1.0* date 2024/8/15 16:33**/ Data NoArgsConstructor AllArgsConstructor Builder public class OracleTableInfo …...
网络性能优化
网络性能优化是确保网络稳定性、速度和可靠性的关键步骤。优化过程通常包括诊断问题、识别瓶颈以及实施具体的解决方案。以下是关于如何进行网络性能优化的详细指南： 一、问题诊断 网络性能监控 网络流量分析工具：使用Wireshark、NetFlow、Ntop等工具监…...
[C++String]接口解读，深拷贝和浅拷贝，string的模拟实现
💖💖💖欢迎来到我的博客，我是anmory💖💖💖 又和大家见面了 欢迎来到C探索系列 作为一个程序员你不能不掌握的知识 先来自我推荐一波 个人网站欢迎访问以及捐款 推荐阅读 如何低成本搭建个人网站…...
理性看待、正确理解 AI 中的 Scaling “laws”
编者按：LLMs 规模和性能的不断提升，让人们不禁产生疑问：这种趋势是否能一直持续下去？我们是否能通过不断扩大模型规模最终实现通用人工智能（AGI）？回答这些问题对于理解 AI 的未来发展轨迹至关重…...