【llama_factory】qwen2_vl训练与批量推理

训练llama factory配置文件

文件：examples/train_lora/qwen2vl_lora_sft.yaml

### model
model_name_or_path: qwen2_vl/model_72b
trust_remote_code: true### method
stage: sft
do_train: true
finetuning_type: lora
lora_target: all### dataset
dataset: car_item  # video: mllm_video_demo
template: qwen2_vl
cutoff_len: 2048
max_samples: 1000
overwrite_cache: true
preprocessing_num_workers: 16### output
output_dir: saves/qwen2_vl-72b/lora/sft
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true### train
per_device_train_batch_size: 1
gradient_accumulation_steps: 1
learning_rate: 1.0e-4
num_train_epochs: 25.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: True
ddp_timeout: 180000000
deepspeed: examples/deepspeed/ds_z3_config.json
#有坑，默认的配置文件没有设置deepspeed参数，会每张显卡都并行加载一个72B模型导致显卡OOM，经过查阅文献设置deepspeed分布式训练参数，有五种，训练大模型设置显存最低的，训练7b可以设置为ds_z0_config.json
#ds_z0_config.json          ds_z2_config.json          ds_z2_offload_config.json  ds_z3_config.json          ds_z3_offload_config.json  
### eval
val_size: 0.1
per_device_eval_batch_size: 0
eval_strategy: steps
eval_steps: 500

llamafactory-cli train  examples/train_lora/qwen2vl_lora_sft.yaml 

批量推理

文件：examples/train_lora/qwen2vl_lora_sft.yaml

### model
model_name_or_path: qwen2_vl/model_7b
trust_remote_code: true
# method
stage: sft
do_train: false
do_predict: true
predict_with_generate: true
finetuning_type: full# dataset
eval_dataset: car_item  #修改为测试集
template: qwen2_vl
cutoff_len: 2048
max_samples: 200
overwrite_cache: true
preprocessing_num_workers: 16
# output
output_dir: saves/qwen2_vl-7b/lora/sft-infer-1 #修改为保存地址
logging_steps: 1
overwrite_output_dir: true# eval
per_device_eval_batch_size: 4
# generation
max_new_tokens: 128
temperature: 0.1
top_k: 1

```bash
llamafactory-cli train  examples/train_lora/qwen2vl_lora_sft.yaml 

# 总结
总的来说,训练时显存不足需要设置分布式训练方式，deepspeed。
推理时，设置eval_dataset eval相关参数。
另外数据集要进行预处理，搞成标准llama处理格式，加载到data/dataset_info.json中```bash
{"identity": {"file_name": "identity.json"},"alpaca_en_demo": {"file_name": "alpaca_en_demo.json"},"alpaca_zh_demo": {"file_name": "alpaca_zh_demo.json"},"glaive_toolcall_en_demo": {"file_name": "glaive_toolcall_en_demo.json","formatting": "sharegpt","columns": {"messages": "conversations","tools": "tools"}},"glaive_toolcall_zh_demo": {"file_name": "glaive_toolcall_zh_demo.json","formatting": "sharegpt","columns": {"messages": "conversations","tools": "tools"}},"car_item": {"file_name": "car_item/train.json","columns": {"images": "image","prompt": "instruction","query": "input","response": "output"}},"mllm_demo": {"file_name": "mllm_demo.json","formatting": "sharegpt","columns": {"messages": "messages","images": "images"},"tags": {"role_tag": "role","content_tag": "content","user_tag": "user","assistant_tag": "assistant"}},"mllm_video_demo": {"file_name": "mllm_video_demo.json","formatting": "sharegpt","columns": {"messages": "messages","videos": "videos"},"tags": {"role_tag": "role","content_tag": "content","user_tag": "user","assistant_tag": "assistant"}},"alpaca_en": {"hf_hub_url": "llamafactory/alpaca_en","ms_hub_url": "llamafactory/alpaca_en","om_hub_url": "HaM/alpaca_en"},"alpaca_zh": {"hf_hub_url": "llamafactory/alpaca_zh","ms_hub_url": "llamafactory/alpaca_zh"},"alpaca_gpt4_en": {"hf_hub_url": "llamafactory/alpaca_gpt4_en","ms_hub_url": "llamafactory/alpaca_gpt4_en"},"alpaca_gpt4_zh": {"hf_hub_url": "llamafactory/alpaca_gpt4_zh","ms_hub_url": "llamafactory/alpaca_gpt4_zh","om_hub_url": "State_Cloud/alpaca-gpt4-data-zh"},"glaive_toolcall_en": {"hf_hub_url": "llamafactory/glaive_toolcall_en","formatting": "sharegpt","columns": {"messages": "conversations","tools": "tools"}},"glaive_toolcall_zh": {"hf_hub_url": "llamafactory/glaive_toolcall_zh","formatting": "sharegpt","columns": {"messages": "conversations","tools": "tools"}},"lima": {"hf_hub_url": "llamafactory/lima","formatting": "sharegpt"},"guanaco": {"hf_hub_url": "JosephusCheung/GuanacoDataset","ms_hub_url": "AI-ModelScope/GuanacoDataset"},"belle_2m": {"hf_hub_url": "BelleGroup/train_2M_CN","ms_hub_url": "AI-ModelScope/train_2M_CN"},"belle_1m": {"hf_hub_url": "BelleGroup/train_1M_CN","ms_hub_url": "AI-ModelScope/train_1M_CN"},"belle_0.5m": {"hf_hub_url": "BelleGroup/train_0.5M_CN","ms_hub_url": "AI-ModelScope/train_0.5M_CN"},"belle_dialog": {"hf_hub_url": "BelleGroup/generated_chat_0.4M","ms_hub_url": "AI-ModelScope/generated_chat_0.4M"},"belle_math": {"hf_hub_url": "BelleGroup/school_math_0.25M","ms_hub_url": "AI-ModelScope/school_math_0.25M"},"belle_multiturn": {"script_url": "belle_multiturn","formatting": "sharegpt"},"ultra_chat": {"script_url": "ultra_chat","formatting": "sharegpt"},"open_platypus": {"hf_hub_url": "garage-bAInd/Open-Platypus","ms_hub_url": "AI-ModelScope/Open-Platypus"},"codealpaca": {"hf_hub_url": "sahil2801/CodeAlpaca-20k","ms_hub_url": "AI-ModelScope/CodeAlpaca-20k"},"alpaca_cot": {"hf_hub_url": "QingyiSi/Alpaca-CoT","ms_hub_url": "AI-ModelScope/Alpaca-CoT"},"openorca": {"hf_hub_url": "Open-Orca/OpenOrca","ms_hub_url": "AI-ModelScope/OpenOrca","columns": {"prompt": "question","response": "response","system": "system_prompt"}},"slimorca": {"hf_hub_url": "Open-Orca/SlimOrca","formatting": "sharegpt"},"mathinstruct": {"hf_hub_url": "TIGER-Lab/MathInstruct","ms_hub_url": "AI-ModelScope/MathInstruct","columns": {"prompt": "instruction","response": "output"}},"firefly": {"hf_hub_url": "YeungNLP/firefly-train-1.1M","columns": {"prompt": "input","response": "target"}},"wikiqa": {"hf_hub_url": "wiki_qa","columns": {"prompt": "question","response": "answer"}},"webqa": {"hf_hub_url": "suolyer/webqa","ms_hub_url": "AI-ModelScope/webqa","columns": {"prompt": "input","response": "output"}},"webnovel": {"hf_hub_url": "zxbsmk/webnovel_cn","ms_hub_url": "AI-ModelScope/webnovel_cn"},"nectar_sft": {"hf_hub_url": "AstraMindAI/SFT-Nectar","ms_hub_url": "AI-ModelScope/SFT-Nectar"},"deepctrl": {"ms_hub_url": "deepctrl/deepctrl-sft-data"},"adgen_train": {"hf_hub_url": "HasturOfficial/adgen","ms_hub_url": "AI-ModelScope/adgen","split": "train","columns": {"prompt": "content","response": "summary"}},"adgen_eval": {"hf_hub_url": "HasturOfficial/adgen","ms_hub_url": "AI-ModelScope/adgen","split": "validation","columns": {"prompt": "content","response": "summary"}},"sharegpt_hyper": {"hf_hub_url": "totally-not-an-llm/sharegpt-hyperfiltered-3k","formatting": "sharegpt"},"sharegpt4": {"hf_hub_url": "shibing624/sharegpt_gpt4","ms_hub_url": "AI-ModelScope/sharegpt_gpt4","formatting": "sharegpt"},"ultrachat_200k": {"hf_hub_url": "HuggingFaceH4/ultrachat_200k","ms_hub_url": "AI-ModelScope/ultrachat_200k","formatting": "sharegpt","columns": {"messages": "messages"},"tags": {"role_tag": "role","content_tag": "content","user_tag": "user","assistant_tag": "assistant"}},"agent_instruct": {"hf_hub_url": "THUDM/AgentInstruct","ms_hub_url": "ZhipuAI/AgentInstruct","formatting": "sharegpt"},"lmsys_chat": {"hf_hub_url": "lmsys/lmsys-chat-1m","ms_hub_url": "AI-ModelScope/lmsys-chat-1m","formatting": "sharegpt","columns": {"messages": "conversation"},"tags": {"role_tag": "role","content_tag": "content","user_tag": "human","assistant_tag": "assistant"}},"evol_instruct": {"hf_hub_url": "WizardLM/WizardLM_evol_instruct_V2_196k","ms_hub_url": "AI-ModelScope/WizardLM_evol_instruct_V2_196k","formatting": "sharegpt"},"glaive_toolcall_100k": {"hf_hub_url": "hiyouga/glaive-function-calling-v2-sharegpt","formatting": "sharegpt","columns": {"messages": "conversations","tools": "tools"}},"cosmopedia": {"hf_hub_url": "HuggingFaceTB/cosmopedia","columns": {"prompt": "prompt","response": "text"}},"stem_zh": {"hf_hub_url": "hfl/stem_zh_instruction"},"ruozhiba_gpt4": {"hf_hub_url": "hfl/ruozhiba_gpt4_turbo"},"neo_sft": {"hf_hub_url": "m-a-p/neo_sft_phase2","formatting": "sharegpt"},"magpie_pro_300k": {"hf_hub_url": "Magpie-Align/Magpie-Pro-300K-Filtered","formatting": "sharegpt"},"magpie_ultra": {"hf_hub_url": "argilla/magpie-ultra-v0.1","columns": {"prompt": "instruction","response": "response"}},"web_instruct": {"hf_hub_url": "TIGER-Lab/WebInstructSub","columns": {"prompt": "question","response": "answer"}},"openo1_sft": {"hf_hub_url": "llamafactory/OpenO1-SFT","ms_hub_url": "llamafactory/OpenO1-SFT","columns": {"prompt": "prompt","response": "response"}},"llava_1k_en": {"hf_hub_url": "BUAADreamer/llava-en-zh-2k","subset": "en","formatting": "sharegpt","columns": {"messages": "messages","images": "images"},"tags": {"role_tag": "role","content_tag": "content","user_tag": "user","assistant_tag": "assistant"}},"llava_1k_zh": {"hf_hub_url": "BUAADreamer/llava-en-zh-2k","subset": "zh","formatting": "sharegpt","columns": {"messages": "messages","images": "images"},"tags": {"role_tag": "role","content_tag": "content","user_tag": "user","assistant_tag": "assistant"}},"llava_150k_en": {"hf_hub_url": "BUAADreamer/llava-en-zh-300k","subset": "en","formatting": "sharegpt","columns": {"messages": "messages","images": "images"},"tags": {"role_tag": "role","content_tag": "content","user_tag": "user","assistant_tag": "assistant"}},"llava_150k_zh": {"hf_hub_url": "BUAADreamer/llava-en-zh-300k","subset": "zh","formatting": "sharegpt","columns": {"messages": "messages","images": "images"},"tags": {"role_tag": "role","content_tag": "content","user_tag": "user","assistant_tag": "assistant"}},"pokemon_cap": {"hf_hub_url": "llamafactory/pokemon-gpt4o-captions","formatting": "sharegpt","columns": {"messages": "conversations","images": "images"}},"mllm_pt_demo": {"hf_hub_url": "BUAADreamer/mllm_pt_demo","formatting": "sharegpt","columns": {"messages": "messages","images": "images"},"tags": {"role_tag": "role","content_tag": "content","user_tag": "user","assistant_tag": "assistant"}},"oasst_de": {"hf_hub_url": "mayflowergmbh/oasst_de"},"dolly_15k_de": {"hf_hub_url": "mayflowergmbh/dolly-15k_de"},"alpaca-gpt4_de": {"hf_hub_url": "mayflowergmbh/alpaca-gpt4_de"},"openschnabeltier_de": {"hf_hub_url": "mayflowergmbh/openschnabeltier_de"},"evol_instruct_de": {"hf_hub_url": "mayflowergmbh/evol-instruct_de"},"dolphin_de": {"hf_hub_url": "mayflowergmbh/dolphin_de"},"booksum_de": {"hf_hub_url": "mayflowergmbh/booksum_de"},"airoboros_de": {"hf_hub_url": "mayflowergmbh/airoboros-3.0_de"},"ultrachat_de": {"hf_hub_url": "mayflowergmbh/ultra-chat_de"},"dpo_en_demo": {"file_name": "dpo_en_demo.json","ranking": true,"formatting": "sharegpt","columns": {"messages": "conversations","chosen": "chosen","rejected": "rejected"}},"dpo_zh_demo": {"file_name": "dpo_zh_demo.json","ranking": true,"formatting": "sharegpt","columns": {"messages": "conversations","chosen": "chosen","rejected": "rejected"}},"dpo_mix_en": {"hf_hub_url": "llamafactory/DPO-En-Zh-20k","subset": "en","ranking": true,"formatting": "sharegpt","columns": {"messages": "conversations","chosen": "chosen","rejected": "rejected"}},"dpo_mix_zh": {"hf_hub_url": "llamafactory/DPO-En-Zh-20k","subset": "zh","ranking": true,"formatting": "sharegpt","columns": {"messages": "conversations","chosen": "chosen","rejected": "rejected"}},"ultrafeedback": {"hf_hub_url": "llamafactory/ultrafeedback_binarized","ms_hub_url": "llamafactory/ultrafeedback_binarized","ranking": true,"columns": {"prompt": "instruction","chosen": "chosen","rejected": "rejected"}},"rlhf_v": {"hf_hub_url": "llamafactory/RLHF-V","ranking": true,"formatting": "sharegpt","columns": {"messages": "conversations","chosen": "chosen","rejected": "rejected","images": "images"}},"vlfeedback": {"hf_hub_url": "Zhihui/VLFeedback","ranking": true,"formatting": "sharegpt","columns": {"messages": "conversations","chosen": "chosen","rejected": "rejected","images": "images"}},"orca_pairs": {"hf_hub_url": "Intel/orca_dpo_pairs","ranking": true,"columns": {"prompt": "question","chosen": "chosen","rejected": "rejected","system": "system"}},"hh_rlhf_en": {"script_url": "hh_rlhf_en","ranking": true,"columns": {"prompt": "instruction","chosen": "chosen","rejected": "rejected","history": "history"}},"nectar_rm": {"hf_hub_url": "AstraMindAI/RLAIF-Nectar","ms_hub_url": "AI-ModelScope/RLAIF-Nectar","ranking": true},"orca_dpo_de": {"hf_hub_url": "mayflowergmbh/intel_orca_dpo_pairs_de","ranking": true},"kto_en_demo": {"file_name": "kto_en_demo.json","formatting": "sharegpt","columns": {"messages": "messages","kto_tag": "label"},"tags": {"role_tag": "role","content_tag": "content","user_tag": "user","assistant_tag": "assistant"}},"kto_mix_en": {"hf_hub_url": "argilla/kto-mix-15k","formatting": "sharegpt","columns": {"messages": "completion","kto_tag": "label"},"tags": {"role_tag": "role","content_tag": "content","user_tag": "user","assistant_tag": "assistant"}},"ultrafeedback_kto": {"hf_hub_url": "argilla/ultrafeedback-binarized-preferences-cleaned-kto","ms_hub_url": "AI-ModelScope/ultrafeedback-binarized-preferences-cleaned-kto","columns": {"prompt": "prompt","response": "completion","kto_tag": "label"}},"wiki_demo": {"file_name": "wiki_demo.txt","columns": {"prompt": "text"}},"c4_demo": {"file_name": "c4_demo.json","columns": {"prompt": "text"}},"refinedweb": {"hf_hub_url": "tiiuae/falcon-refinedweb","columns": {"prompt": "content"}},"redpajama_v2": {"hf_hub_url": "togethercomputer/RedPajama-Data-V2","columns": {"prompt": "raw_content"},"subset": "default"},"wikipedia_en": {"hf_hub_url": "olm/olm-wikipedia-20221220","ms_hub_url": "AI-ModelScope/olm-wikipedia-20221220","columns": {"prompt": "text"}},"wikipedia_zh": {"hf_hub_url": "pleisto/wikipedia-cn-20230720-filtered","ms_hub_url": "AI-ModelScope/wikipedia-cn-20230720-filtered","columns": {"prompt": "completion"}},"pile": {"hf_hub_url": "monology/pile-uncopyrighted","ms_hub_url": "AI-ModelScope/pile","columns": {"prompt": "text"}},"skypile": {"hf_hub_url": "Skywork/SkyPile-150B","ms_hub_url": "AI-ModelScope/SkyPile-150B","columns": {"prompt": "text"}},"fineweb": {"hf_hub_url": "HuggingFaceFW/fineweb","columns": {"prompt": "text"}},"fineweb_edu": {"hf_hub_url": "HuggingFaceFW/fineweb-edu","columns": {"prompt": "text"}},"the_stack": {"hf_hub_url": "bigcode/the-stack","ms_hub_url": "AI-ModelScope/the-stack","columns": {"prompt": "content"}},"starcoder_python": {"hf_hub_url": "bigcode/starcoderdata","ms_hub_url": "AI-ModelScope/starcoderdata","columns": {"prompt": "content"},"folder": "python"}
}