在昇腾训练平台上适配Hunyuan3D 2.0 模型的推理

在昇腾训练平台上适配Hunyuan3D 2.0 模型的推理【免费下载链接】cann-recipes-embodied-intelligence本项目针对具身智能业务中的典型模型、加速算法提供基于CANN平台的优化样例项目地址: https://gitcode.com/cann/cann-recipes-embodied-intelligenceHunyuan3D模型是腾讯混元系列在2025年推出的一款3D资产创作模型用于生成带有高分辨率纹理贴图的高保真度3D模型本项目旨在提供Hunyuan3D的NPU适配版本方便用户能够在昇腾生态上直接使用Hunyuan3D。此外本样例基于Hunyuan3D模型在NPU进行了性能优化目前texgen在2万平面mesh网格输入下推理时间降至26秒。详细内容可至性能优化章节进行查看。执行样例环境准备本样例采用CANN 8.2.RC1。请从CANN软件包下载地址下载Ascend-cann-toolkit_${version}_linux-${arch}.run与Ascend-cann-kernels-${chip_type}_${version}_linux-${arch}.run软件包并参考CANN安装文档进行安装。conda create -n hunyuan3d python3.10.18 conda activate hunyuan3d本样例的torch以及torch_npu版本为2.6请从Ascend Extension for PyTorch插件下载torch与torch_npu安装包同时指定decorator版本pip install torch2.6.0 pip install torchvision0.21.0 pip install torch-npu2.6.0.post3 pip install decorator5.2.1 pip install ninja网络模型代码准备本仓库依赖Hunyuan3D的开源仓库代码。进入Hunyuan3D的官方仓库下载Hunyuan3D模型网络结构代码git clone https://github.com/Tencent-Hunyuan/Hunyuan3D-2.git下载本仓库代码git clone https://gitcode.com/cann/cann-recipes-embodied-intelligence.git将Hunyuan3D仓库的网络模型文件以非覆盖模式复制到本项目目录下。其中下方命令里的/path/to/Hunyuan3D-2/改为Hunyuan3D-2文件路径。cd cann-recipes-embodied-intelligence/3d_vision/Hunyuan3D cp -rn /path/to/Hunyuan3D-2/* ./安装Python依赖pip install -r requirements.txt编译第三方代码cd hy3dgen/texgen/differentiable_renderer python3 setup.py install模型结构修改部分如下所示Hunyuan3D --- hy3dgen --- texgen --- custom_rasterizer custom_rasterizer in current depository #替换当前文件 --- differentiable_renderer --- mesh_render.py differentiable_renderer/mesh_render.py #替换文件 --- rasterizer.py #添加新文件 --- minimal_demo_npu.py #添加新文件权重模型NPU环境支持Hunyuan3D模型列表用户可根据需要下载蒸馏版本Hunyuan3D-2mv 系列ModelDescriptionDateSizeHuggingfaceHunyuan3D-DiT-v2-mvMultiview Image to Shape Model2025-03-181.1BDownloadHunyuan3D-2 系列ModelDescriptionDateSizeHuggingfaceHunyuan3D-DiT-v2-0Image to Shape Model2025-01-211.1BDownloadHunyuan3D-Paint-v2-0Texture Generation Model2025-01-211.3BDownloadHunyuan3D-Delight-v2-0Image Delight Model2025-01-211.3BDownload权重安装地址为base_dirmodel_path默认base_dir为~/.cache/hy3dgen/如需修改base_dir需将./hy3dgen/shapegen/utils.py中第97行和./hy3dgen/texgen/pipelines.py中第61行进行修改class Hunyuan3DPaintPipeline: classmethod def from_pretrained(cls, model_path, subfolderhunyuan3d-paint-v2-0-turbo): original_model_path model_path if not os.path.exists(model_path): # try local path base_dir os.environ.get(HY3DGEN_MODELS, ~/.cache/hy3dgen/) model_path os.path.expanduser(os.path.join(base_dir, model_path))最后模型权重路径为base_dirmodel_path --- hunyuan3d-dit-v2-0 #单视角图像DIT模型 --- hunyuan3d-dit-v2-mv #多视角图像DIT模型 --- hunyuan3d-delight-v2-0 --- hunyuan3d-paint-v2-0执行推理执行推理由于本项目位于3d_vision/Hunyuan3D目录下并依靠仓库根目录下module/提供的公共优化模块因此在开始推理前需要设置环境变量执行如下指令source set_env.sh即可将当前样例目录及仓库根目录加入环境变量中之后执行python minimal_demo_npu.pyminimal_demo_npu.py采用默认设置执行单图像推理输入图片默认读取当前目录下的assets/demo.png生成结果默认输出到当前目录下的outputs/。运行不同配置可参考以下脚本执行python minimal_demo_npu.py --model_path tencent/Hunyuan3D-2 --multiview --face_reduce --full_graph --multi_thread (--use_render_npu) --save_rendermodel_path选择模型路径multiview设置是否采用多视角推理face_reduce设置是否减少三角面片full_graph设置是否采用图模式multi_thread设置是否采用多线程并行执行光栅化use_render_npu设置是否采用npu方式执行光栅化不能与multi_thread共同使用save_render设置是否复用光栅化结果。推理步骤介绍Hunyuan3D源码参考diffusers接口API用于物体生成包括Hunyuan3D-DiT和Hunyuan3D-Paint两个部分。运行Hunyuan3D-DiT方式如下from hy3dgen.shapegen import Hunyuan3DDiTFlowMatchingPipeline from torch_npu.contrib import transfer_to_npu pipeline Hunyuan3DDiTFlowMatchingPipeline.from_pretrained(tencent/Hunyuan3D-2) mesh pipeline(imageassets/demo.png)[0]运行Hunyuan3D-Paint方式如下from hy3dgen.texgen import Hunyuan3DPaintPipeline from hy3dgen.shapegen import Hunyuan3DDiTFlowMatchingPipeline from torch_npu.contrib import transfer_to_npu pipeline Hunyuan3DDiTFlowMatchingPipeline.from_pretrained(tencent/Hunyuan3D-2) mesh pipeline(imageassets/demo.png)[0] pipeline Hunyuan3DPaintPipeline.from_pretrained(tencent/Hunyuan3D-2) mesh pipeline(mesh, imageassets/demo.png)Hunyuan3D-DiT执行过程中我们在NPU环境下提供多视角模型推理具体方式如下from hy3dgen.texgen import Hunyuan3DPaintPipeline from hy3dgen.shapegen import Hunyuan3DDiTFlowMatchingPipeline from torch_npu.contrib import transfer_to_npu images { front: assets/example_mv_images/1/front.png, left: assets/example_mv_images/1/left.png, back: assets/example_mv_images/1/back.png } pipeline Hunyuan3DDiTFlowMatchingPipeline.from_pretrained( tencent/Hunyuan3D-2mv, subfolderhunyuan3d-dit-v2-mv, variantfp16 ) mesh pipeline( imageimages, num_inference_steps50, octree_resolution380, num_chunks20000, generatortorch.manual_seed(12345), output_typetrimesh )[0] pipeline Hunyuan3DPaintPipeline.from_pretrained(tencent/Hunyuan3D-2) mesh pipeline(mesh, imagelist(image.values()))Citationmisc{lai2025hunyuan3d25highfidelity3d, title{Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate Details}, author{Tencent Hunyuan3D Team}, year{2025}, eprint{2506.16504}, archivePrefix{arXiv}, primaryClass{cs.CV}, url{https://arxiv.org/abs/2506.16504}, } misc{hunyuan3d22025tencent, title{Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation}, author{Tencent Hunyuan3D Team}, year{2025}, eprint{2501.12202}, archivePrefix{arXiv}, primaryClass{cs.CV} } misc{yang2024hunyuan3d, title{Hunyuan3D 1.0: A Unified Framework for Text-to-3D and Image-to-3D Generation}, author{Tencent Hunyuan3D Team}, year{2024}, eprint{2411.02293}, archivePrefix{arXiv}, primaryClass{cs.CV} }【免费下载链接】cann-recipes-embodied-intelligence本项目针对具身智能业务中的典型模型、加速算法提供基于CANN平台的优化样例项目地址: https://gitcode.com/cann/cann-recipes-embodied-intelligence创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考