【ms-swift 大模型微调实战】

安装环境

• pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
• pip install modelscope vllm ‘ms-swift[llm]’ -U

下载模型

• modelscope download --model Qwen/Qwen2.5-7B-Instruct --local_dir ./Qwen2.5-7B-Instruct

微调

• 实验环境：4 * A100# 显存占用：4 * 70GB

    NPROC_PER_NODE=4 CUDA_VISIBLE_DEVICES=0,1,2,3 swift sft \--model_type qwen2_5-72b-instruct \--model_id_or_path Qwen2.5-72B-Instruct \--dataset qwen2-pro-en#500 qwen2-pro-zh#500 self-cognition#500 \--logging_steps 5 \--learning_rate 1e-4 \--output_dir output \ --lora_target_modules ALL \--model_name 小黄 'Xiao Huang' \--model_author 魔搭 ModelScope \--deepspeed default-zero3

• 单卡A10/3090可运行的例子 （Qwen2.5-7B-Instruct）# 显存占用：24GB

  	CUDA_VISIBLE_DEVICES=0 swift sft \--model_type qwen2_5-7b-instruct \--model_id_or_path Qwen2.5-7B-Instruct \--dataset qwen2-pro-en#500 qwen2-pro-zh#500 self-cognition#500 \--logging_steps 5 \--max_length 2048 \--learning_rate 1e-4 \--output_dir output \--lora_target_modules ALL \--model_name 小黄 'Xiao Huang' \--model_author 魔搭 ModelScope

融合lora

CUDA_VISIBLE_DEVICES=0,1 swift export \    --ckpt_dir output/qwen2_5-72b-instruct/vx-xxx/checkpoint-xxx \    --merge_lora true

vLLM部署

vllm serve xxxxx-checkpoint-merged [opentional args]

参考

• Qwen2.5 全链路模型体验、下载、推理、微调、部署实战！
• https://github.com/modelscope/ms-swift
• 自定义数据集
• 微调文档
• 推理文档