在本地Windows机器加载大模型并生成内容

本篇演示在本地机器下载和加载大模型并获取AI产生的内容。简单起见，使用的大模型是Qwen2.5-0.5B-Instruct，整个模型的所有文件不到1G。

Qwen2.5-0.5B-Instruct 是阿里巴巴云 QWen 团队基于 Transformer 架构开发的轻量级指令调优语言模型，专为资源有限场景设计。

环境准备

• OS： Windows 10
• 开发IDE ： VS Code
• Python： 3.11

Python库安装步骤：

1. 安装TensorFlow

pip install tensorflow

TensorFlow 是 Google 开发的开源深度学习框架，提供灵活高效的模型构建、训练与部署能力，支持多平台应用与大规模机器学习任务。

2. 安装PyTorch

pip install torch

PyTorch 是由 Facebook 人工智能研究院（FAIR）研发的开源深度学习框架，以动态计算图、易用接口和强大的自动求导功能为核心，助力快速迭代研究与灵活模型开发。

3. 安装accelerate

pip install accelerate

Accelerate 是 Hugging Face 推出的深度学习优化库，通过简化分布式训练、混合精度计算及硬件加速（如 GPU/TPU）的集成，显著提升模型训练效率与资源利用率。

4. 安装modelscope

ModelScope 的 Python 库是阿里开发的一个功能强大的工具，旨在帮助开发者更方便地使用和管理机器学习模型，尤其是预训练模型。

编写代码

这里的代码是让 Qwen2.5-0.5B-Instruct 完成一个Java的闰年计算的编码任务：

from modelscope import AutoModelForCausalLM, AutoTokenizermodel_name = "Qwen/Qwen2.5-0.5B-Instruct"model = AutoModelForCausalLM.from_pretrained(model_name,torch_dtype="auto",device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)input_text = "请完成一个Java的计算闰年的函数，不需要任何解释性的内容，只需要输出代码就可以"
messages = [{"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},{"role": "user", "content": input_text}
]
text = tokenizer.apply_chat_template(messages,tokenize=False,add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)# Use `max_new_tokens` to control the maximum output length.
generated_ids = model.generate(model_inputs.input_ids, max_new_tokens=1024, do_sample=False)[0]
# The generated_ids include prompt_ids, so we only need to decode the tokens after prompt_ids.
output_text = tokenizer.decode(generated_ids[len(model_inputs.input_ids[0]):], skip_special_tokens=True)print(f"Prompt: {input_text}\n\nGenerated text: {output_text}")

• 代码运行首先会下载模型文件
• 下载完成之后就会加载模型并且生成结果了。

执行后的结果如下：

问题解决

如果不是严格按照上述步骤执行的话，运行过程的可能会有如下问题，可以参照解决。

1 . cannot import name ‘AutoModelForCausalLM’ from ‘modelscope’

详细的错误信息如下：

PS D:\devworkspace\vs\ai_ency\ai_python_ency> & d:/devworkspace/vs/ai_ency/ai_python_ency/.conda/python.exe d:/devworkspace/vs/ai_ency/ai_python_ency/qwen/quickstart/Qwen2.5-Coder-0.5B.py
Traceback (most recent call last):File "d:\devworkspace\vs\ai_ency\ai_python_ency\qwen\quickstart\Qwen2.5-Coder-0.5B.py", line 1, in <module>from modelscope import AutoModelForCausalLM, AutoTokenizer
ImportError: cannot import name 'AutoModelForCausalLM' from 'modelscope' (D:\devworkspace\vs\ai_ency\ai_python_ency\.conda\Lib\site-packages\modelscope\__init__.py)    

这个问题比较难发现，错误信息的提示比较不明确， 原因其实是没有安装 TensorFlow 或者TensorFlow 的版本过低。

解决方法，安装会升级TensorFlow

2.None of PyTorch, TensorFlow >= 2.0, or Flax have been found.

PS D:\devworkspace\vs\ai_ency\ai_python_ency> & d:/devworkspace/vs/ai_ency/ai_python_ency/.conda/python.exe d:/devworkspace/vs/ai_ency/ai_python_ency/qwen/quickstart/Qwen2.5-Coder-0.5B.py
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Traceback (most recent call last):File "d:\devworkspace\vs\ai_ency\ai_python_ency\qwen\quickstart\Qwen2.5-Coder-0.5B.py", line 2, in <module>from modelscope import AutoModelForCausalLM, AutoTokenizer
ImportError: cannot import name 'AutoModelForCausalLM' from 'modelscope' (D:\devworkspace\vs\ai_ency\ai_python_ency\.conda\Lib\site-packages\modelscope\__init__.py)   

这里同样是有 ImportError: cannot import name ‘AutoModelForCausalLM’ 的提示，但是同时会提示 None of PyTorch， 很明显就是缺少了PyTorch。

注意，安装的命令是：

pip install torch

3.ImportError: Using low_cpu_mem_usage=True, a device_map or a tp_plan requires Accelerate: pip install 'accelerate>=0.26.0'

Downloading Model from https://www.modelscope.cn to directory: D:\ai\modelscope\cache\models\Qwen\Qwen2.5-0.5B-Instruct
2025-03-25 21:11:49,487 - modelscope - WARNING - Using branch: master as version is unstable, use with caution
2025-03-25 21:11:49,961 - modelscope - INFO - Creating symbolic link [D:\ai\modelscope\cache\models\Qwen\Qwen2.5-0.5B-Instruct].
2025-03-25 21:11:49,962 - modelscope - WARNING - Failed to create symbolic link D:\ai\modelscope\cache\models\Qwen\Qwen2.5-0.5B-Instruct for D:\ai\modelscope\cache\models\Qwen\Qwen2.5-0.5B-Instruct\Qwen\Qwen2___5-0___5B-Instruct.
Traceback (most recent call last):File "d:\devworkspace\vs\ai_ency\ai_python_ency\qwen\quickstart\Qwen2.5-Coder-0.5B.py", line 6, in <module>model = AutoModelForCausalLM.from_pretrained(^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "D:\devworkspace\vs\ai_ency\ai_python_ency\.conda\Lib\site-packages\modelscope\utils\hf_util\patcher.py", line 230, in from_pretrainedmodule_obj = module_class.from_pretrained(^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "D:\devworkspace\vs\ai_ency\ai_python_ency\.conda\Lib\site-packages\transformers\models\auto\auto_factory.py", line 573, in from_pretrainedreturn model_class.from_pretrained(^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "D:\devworkspace\vs\ai_ency\ai_python_ency\.conda\Lib\site-packages\transformers\modeling_utils.py", line 272, in _wrapperreturn func(*args, **kwargs)^^^^^^^^^^^^^^^^^^^^^File "D:\devworkspace\vs\ai_ency\ai_python_ency\.conda\Lib\site-packages\transformers\modeling_utils.py", line 4203, in from_pretrainedraise ImportError(
ImportError: Using `low_cpu_mem_usage=True`, a `device_map` or a `tp_plan` requires Accelerate: `pip install 'accelerate>=0.26.0'`

这个提示也很明显，安装accelerate 就可以了。

---