Deepspeed : AttributeError: ‘DummyOptim‘ object has no attribute ‘step‘

题意：尝试在一个名为 DummyOptim 的对象上调用 .step() 方法，但是这个对象并没有定义这个方法

问题背景：

I want to use deepspeed for training LLMs along with Huggingface Trainer. But when I use deepspeed along with trainer I get error "AttributeError: 'DummyOptim' object has no attribute 'step'". Below is my code

尝试结合使用 DeepSpeed 和 Hugging Face 的 Trainer API 来训练大型语言模型（LLMs）时遇到 "AttributeError: 'DummyOptim' object has no attribute 'step'" 这个错误，下面是我的代码：

import argparse
import numpy as np
import torch
from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForCausalLMfrom trl import DPOTrainer, DPOConfig
def preprocess_data(item):return {'prompt': 'Instruct: ' + item['prompt'] + '\n','chosen': 'Output: ' + item['chosen'],'rejected': 'Output: ' + item['rejected']}        def main():parser = argparse.ArgumentParser()parser.add_argument("--epochs", type=int, default=1)parser.add_argument("--beta", type=float, default=0.1)parser.add_argument("--batch_size", type=int, default=4)parser.add_argument("--lr", type=float, default=1e-6)parser.add_argument("--seed", type=int, default=2003)parser.add_argument("--model_name", type=str, default="EleutherAI/pythia-14m")parser.add_argument("--dataset_name", type=str, default="jondurbin/truthy-dpo-v0.1")parser.add_argument("--local_rank", type=int, default=0)args = parser.parse_args()# Determine device based on local_rankdevice = torch.device("cuda", args.local_rank) if torch.cuda.is_available() else torch.device("cpu")tokenizer = AutoTokenizer.from_pretrained(args.model_name)tokenizer.pad_token = tokenizer.eos_tokenmodel = AutoModelForCausalLM.from_pretrained(args.model_name).to(device)ref_model = AutoModelForCausalLM.from_pretrained(args.model_name).to(device)dataset = load_dataset(args.dataset_name, split="train")dataset = dataset.map(preprocess_data)# Split the dataset into training and validation setsdataset = dataset.train_test_split(test_size=0.1, seed=args.seed)train_dataset = dataset['train']val_dataset = dataset['test']training_args = DPOConfig(learning_rate=args.lr,num_train_epochs=args.epochs,per_device_train_batch_size=args.batch_size,logging_steps=10,remove_unused_columns=False,max_length=1024,max_prompt_length=512,deepspeed="ds_config.json"       )# Verify and print embedding dimensions before finetuningprint("Base model embedding dimension:", model.config.hidden_size)model.train()ref_model.eval()dpo_trainer = DPOTrainer(model,ref_model,beta=args.beta,train_dataset=train_dataset,eval_dataset=val_dataset,tokenizer=tokenizer,args=training_args,)dpo_trainer.train()# Evaluateevaluation_results = dpo_trainer.evaluate()print("Evaluation Results:", evaluation_results)save_model_name = 'finetuned_model'model.save_pretrained(save_model_name)if __name__ == "__main__":main()

The config file used is the below one        使用的配置文件是下面的这个：

{
"zero_optimization": {"stage": 3,"offload_optimizer": {"device": "cpu","pin_memory": true},"offload_param": {"device": "cpu","pin_memory": true},"overlap_comm": true,"contiguous_gradients": true,"sub_group_size": 1e9,"reduce_bucket_size": "auto","stage3_prefetch_bucket_size": "auto","stage3_param_persistence_threshold": "auto","stage3_max_live_parameters": 1e9,"stage3_max_reuse_distance": 1e9,"stage3_gather_16bit_weights_on_model_save": true},
"bf16": {"enabled": "auto"
},
"fp16": {"enabled": "auto","loss_scale": 0,"initial_scale_power": 32,"loss_scale_window": 1000,"hysteresis": 2,"min_loss_scale": 1
},"gradient_accumulation_steps": "auto",
"gradient_clipping": "auto",
"train_batch_size": "auto",
"train_micro_batch_size_per_gpu": "auto",
"wall_clock_breakdown": false,
"flops_profiler": {"enabled": false,"detailed": false
},
"optimizer": {"type": "Lamb","params": {"lr": "auto","betas": [0.9, 0.999],"eps": "auto","weight_decay": "auto"}
},
"zero_allow_untested_optimizer": true
}

The code works with out deepspeed. I have torch=2.3.1, deepspeed =0.14.5, trl=0.9.4 and CUDA Version: 12.5.

在没有使用 DeepSpeed 的情况下，代码可以正常工作。当前的软件版本配置为：PyTorch 2.3.1，DeepSpeed 0.14.5，TRL 0.9.4，以及 CUDA 版本 12.5。

Appreciate any hint on this !        非常感谢您在这方面的任何提示！

问题解决：

from accelerate.utils import DistributedTypetraining_args.distributed_state.distributed_type = DistributedType.DEEPSPEED

adding this solves the issue        添加这个解决了问题