AI智算-k8s部署DeepSeek Janus-Pro-7B 多模态大模型

简介

DeepSeek Janus Pro 作为一款强大的多模态理解与生成框架，正在成为研究人员和开发者的热门选择。本文将详细介绍如何在云原生k8s环境中部署配置和使用 DeepSeek Janus Pro，以及它在多模态任务中的强大功能。

环境依赖

Kubernetes、GPU（H100）

模型下载

参考链接：https://www.modelscope.cn/models/deepseek-ai/Janus-Pro-7B/files

modelscope download --model deepseek-ai/Janus-Pro-7B --local_dir file_CPU_01/modelServing/Janus-Pro-7B/

下载Janus库

git clone  https://github.com/deepseek-ai/Janus.git

GPU环境镜像

modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.1.0-py310-torch2.3.1-tf2.16.1-1.22.2

模型manifest

cat Janus-Pro.yaml 
kind: Deployment
apiVersion: apps/v1
metadata:name: janusprolabels:app: januspro
spec:replicas: 1selector:matchLabels:app: janusprotemplate:metadata:labels:app: janusprospec:schedulerName: volcanovolumes:- name: shmemptyDir:medium: MemorysizeLimit: 512G- name: host-timehostPath:path: /etc/localtimetype: ''- name: model-volumehostPath:path: /file_CPU_01/modelServing/Janus-Pro-7B/type: ''containers:- name: janusproimage: modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.1.0-py310-torch2.3.1-tf2.16.1-1.22.2command: ["/bin/bash", "-c"]args: ["/modelscope_cache/startup.sh"]ports:- name: http-8000containerPort: 8000protocol: TCPenv:- name: MODELSCOPE_CACHEvalue: "/modelscope_cache"resources:limits:nvidia.com/gpu: '8'requests:nvidia.com/gpu: '8'volumeMounts:- name: host-timereadOnly: truemountPath: /etc/localtime- name: shmmountPath: /dev/shm- name: model-volumemountPath: /modelscope_cacheimagePullPolicy: IfNotPresentrestartPolicy: Always
---
kind: Service
apiVersion: v1
metadata:name: janusprolabels:app: januspro
spec:ports:- name: http-7860protocol: TCPport: 7860targetPort: 7860selector:app: janusprotype: NodePort

###模型启动脚本###
cat startup.sh 
#!/bin/bash
# startup.sh# 复制文件
cp /modelscope_cache/Janus-main/frpc_linux_amd64_v0.3 /usr/local/lib/python3.10/site-packages/gradio/frpc_linux_amd64_v0.3# 安装依赖
echo "Installing dependencies..."
cd /modelscope_cache/Janus-main
pip install -e . # 启动
python demo/app_januspro.py --device cuda

调用Janus

kubectl get svc -n sre-tools 
NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)           AGE
januspro             NodePort    10.233.38.3     <none>        7860:31456/TCP    3h52m

用宿主机$ip:31456 访问即可

多模态

文生图

PS：
ModelScope server无法直接加载本地下载好的Janus-Pro模型