AI智算-k8s部署大语言模型管理工具Ollama

简介

Github：https://github.com/ollama/ollama
官网：https://ollama.com/
API：https://github.com/ollama/ollama/blob/main/docs/api.md

Ollama 是一个基于 Go 语言开发的可以本地运行大模型的开源框架，同时提供 REST API 管理和使用大模型。

k8s部署Ollama

~# cat ollama-dp.yaml 
kind: Deployment
apiVersion: apps/v1
metadata:name: ollamalabels:app: ollama
spec:replicas: 1selector:matchLabels:app: ollamatemplate:metadata:labels:app: ollamaspec:volumes:- name: ollama-modelshostPath:path: /file_CPU_01/cyw_data/ollamatype: ''- name: host-timehostPath:path: /etc/localtimetype: ''containers:- name: ollamaimage: ollama/ollama:0.5.1ports:- name: http-11434containerPort: 11434protocol: TCPresources:limits:nvidia.com/gpu: '8'requests:nvidia.com/gpu: '8'volumeMounts:- name: ollama-modelsmountPath: /root/.ollama- name: host-timereadOnly: truemountPath: /etc/localtimeimagePullPolicy: IfNotPresentrestartPolicy: AlwaysschedulerName: volcano
---
kind: Service
apiVersion: v1
metadata:name: ollamalabels:app: ollama
spec:ports:- name: http-11434protocol: TCPport: 11434targetPort: 11434nodePort: 31434selector:app: ollamatype: NodePort

~# kubectl get pods -n sre-tools |grep ollama
ollama-8cd499f48-4b6wv      1/1     Running   0          13m
~# kubectl get svc -n sre-tools |grep ollama
ollama   NodePort   10.233.6.148   <none>        11434:31434/TCP   13m

测试接口：

~# curl -i  http://10.0.xx.xx:31434
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Date: Thu, 12 Dec 2024 08:46:55 GMT
Content-Length: 17Ollama is running

Open WebUI

Github：https://github.com/open-webui/open-webui
官网：https://openwebui.com/

Open WebUI 是一个可扩展、功能丰富且用户友好的自托管 AI 界面，旨在完全离线运行。它支持各种 LLM 运行器，包括 Ollama 和 OpenAI 兼容 API。

~# cat open-webui.yaml 
apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: open-webui-data-pvc
spec:accessModes:- ReadWriteOnceresources:requests:storage: 20GistorageClassName: nfs-client---
apiVersion: apps/v1
kind: Deployment
metadata:name: open-webui-deployment
spec:replicas: 1selector:matchLabels:app: open-webuitemplate:metadata:labels:app: open-webuispec:containers:- name: open-webuiimage: ghcr.mirrorify.net/open-webui/open-webui:mainports:- containerPort: 8080env:- name: OLLAMA_BASE_URLvalue: "http://10.0.xx.xx:31434"   #ollama地址volumeMounts:- name: open-webui-datamountPath: /app/backend/datavolumes:- name: open-webui-datapersistentVolumeClaim:claimName: open-webui-data-pvc---
apiVersion: v1
kind: Service
metadata:name: open-webui-service
spec:type: ClusterIPports:- port: 3000targetPort: 8080 selector:app: open-webui---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: open-webui-ingress
spec:rules:- host: open-webui.xxx-sh.comhttp:paths:- backend:service:name: open-webui-serviceport:number: 3000path: /pathType: Prefixtls:- hosts:- open-webui.xxx-sh.comsecretName: xxx-tls

访问Open-WebUI

https://open-webui.xxx-sh.com