当前位置： 首页 > article >正文

PyTorch 深度学习实战（17）：Asynchronous Advantage Actor-Critic (A3C) 算法与并行训练

article 2026/3/5 8:54:10

在上一篇文章中，我们深入探讨了 Soft Actor-Critic (SAC) 算法及其在平衡探索与利用方面的优势。本文将介绍强化学习领域的重要里程碑——Asynchronous Advantage Actor-Critic (A3C) 算法，并展示如何利用 PyTorch 实现并行化训练来加速学习过程。

一、A3C 算法原理

A3C 算法由 DeepMind 于 2016 年提出，通过异步并行的多个智能体（Worker）与环境交互，显著提升了训练效率。其核心思想可概括为：

  1. 并行架构 多个 Worker 线程同时与各自的环境副本交互，收集经验并计算梯度。

  2. 参数共享 所有 Worker 共享全局网络参数，定期将本地梯度同步到全局网络。

  3. 优势函数（Advantage） 使用 A(s,a)=Q(s,a)−V(s)A(s,a)=Q(s,a)−V(s) 衡量动作的相对优势，降低方差。

算法优势

  • 高效数据收集：并行交互突破数据相关性限制

  • 稳定训练：异步更新缓解梯度冲突

  • 资源利用率高：充分利用多核 CPU 资源

二、A3C 实现步骤

我们将使用 PyTorch 实现 A3C 算法解决 CartPole 平衡问题：

  1. 定义全局网络结构 包含 Actor 和 Critic 两个分支

  2. 实现异步 Worker 每个 Worker 独立与环境交互并计算梯度

  3. 设计并行更新机制 异步更新全局网络参数

  4. 训练与评估

三、代码实现

import torch
import torch.nn as nn
import torch.optim as optim
import torch.multiprocessing as mp
import gym
​
# 定义全局网络
class GlobalNetwork(nn.Module):def __init__(self, state_dim, action_dim):super().__init__()self.base = nn.Sequential(nn.Linear(state_dim, 128),nn.ReLU())self.actor = nn.Linear(128, action_dim)self.critic = nn.Linear(128, 1)
​def forward(self, x):x = self.base(x)return torch.softmax(self.actor(x), dim=-1), self.critic(x)
​
# Worker 进程定义
class Worker(mp.Process):def __init__(self, global_net, optimizer, global_ep, name):super().__init__()self.env = gym.make('CartPole-v1')self.local_net = GlobalNetwork(4, 2)  # 本地网络self.global_net = global_netself.optimizer = optimizerself.global_ep = global_epself.name = name
​def run(self):while self.global_ep.value < 300:  # 训练300个全局episode# 同步全局参数到本地self.local_net.load_state_dict(self.global_net.state_dict())# 收集轨迹数据states, actions, rewards = [], [], []state, _ = self.env.reset()  # 正确解包observationep_reward = 0while True:state_tensor = torch.FloatTensor(state).unsqueeze(0)  # 添加批次维度 [1,4]prob, _ = self.local_net(state_tensor)action = torch.multinomial(prob, 1).item()next_state, reward, done, _, _ = self.env.step(action)  # 解包所有返回值ep_reward += rewardstates.append(state)actions.append(action)rewards.append(reward)if done:# 计算优势函数returns, advantages = self.cal_advantages(states, rewards)# 异步更新全局网络self.update_global(states, actions, advantages, returns)with self.global_ep.get_lock():self.global_ep.value += 1print(f"Episode {self.global_ep.value} finished by {self.name} with reward {ep_reward}")breakstate = next_state
​def cal_advantages(self, states, rewards):# 计算回报returns = []discounted_reward = 0for r in reversed(rewards):discounted_reward = r + 0.99 * discounted_rewardreturns.insert(0, discounted_reward)# 转换为张量并标准化returns = torch.FloatTensor(returns)returns = (returns - returns.mean()) / (returns.std() + 1e-8)# 计算价值估计states_tensor = torch.FloatTensor(states)  # 形状 [n_steps, 4]_, values = self.local_net(states_tensor)values = values.squeeze().detach()# 计算优势函数advantages = returns - valuesreturn returns, advantages
​def update_global(self, states, actions, advantages, returns):states_tensor = torch.FloatTensor(states)  # 形状 [n_steps, 4]actions_tensor = torch.LongTensor(actions)advantages = advantages.detach()returns = returns.detach()# 计算损失probs, values = self.local_net(states_tensor)policy_loss = -torch.log(probs.gather(1, actions_tensor.unsqueeze(1))) * advantages.unsqueeze(1)policy_loss = policy_loss.mean()value_loss = nn.MSELoss()(values.squeeze(), returns)entropy = -torch.sum(probs * torch.log(probs), dim=1).mean()total_loss = policy_loss + 0.5 * value_loss - 0.01 * entropy# 反向传播并更新全局网络self.optimizer.zero_grad()total_loss.backward()# 将本地梯度上传到全局网络for local_param, global_param in zip(self.local_net.parameters(), self.global_net.parameters()):if global_param.grad is not None:global_param.grad = local_param.grad.clone()self.optimizer.step()
​
if __name__ == '__main__':# 初始化全局网络global_net = GlobalNetwork(4, 2)global_net.share_memory()optimizer = optim.Adam(global_net.parameters(), lr=0.0002)global_ep = mp.Value('i', 0)# 启动4个Worker进程workers = [Worker(global_net, optimizer, global_ep, f"Worker-{i}") for i in range(4)][w.start() for w in workers][w.join() for w in workers]

四、关键代码解析

  1. 网络结构设计

    • 共享特征提取层（base）

    • Actor 输出动作概率分布

    • Critic 评估状态价值

  2. 并行训练机制

    • 使用 torch.multiprocessing 实现多进程

    • share_memory() 实现参数共享

    • 原子计数器 global_ep 控制训练进度

  3. 优势函数计算 使用 N-step 回报计算优势：

五、训练结果

运行代码将观察到：

  1. 多个 Worker 并行训练，控制台输出各进程的即时奖励

  2. 全局训练 episode 达到 300 时自动终止

    Episode 1 finished by Worker-1 with reward 34.0
Episode 2 finished by Worker-2 with reward 60.0
Episode 3 finished by Worker-0 with reward 38.0
Episode 4 finished by Worker-3 with reward 46.0
Episode 5 finished by Worker-2 with reward 17.0
Episode 6 finished by Worker-1 with reward 40.0
Episode 7 finished by Worker-3 with reward 56.0
Episode 8 finished by Worker-0 with reward 54.0
Episode 9 finished by Worker-1 with reward 18.0
Episode 10 finished by Worker-2 with reward 59.0
Episode 11 finished by Worker-3 with reward 19.0
Episode 12 finished by Worker-2 with reward 17.0
Episode 13 finished by Worker-0 with reward 32.0
Episode 14 finished by Worker-3 with reward 18.0
Episode 15 finished by Worker-1 with reward 59.0
Episode 16 finished by Worker-1 with reward 19.0
Episode 17 finished by Worker-3 with reward 15.0
Episode 18 finished by Worker-2 with reward 39.0
Episode 19 finished by Worker-1 with reward 12.0
Episode 20 finished by Worker-0 with reward 29.0
Episode 21 finished by Worker-3 with reward 25.0
Episode 22 finished by Worker-2 with reward 14.0
Episode 23 finished by Worker-1 with reward 17.0
Episode 24 finished by Worker-0 with reward 31.0
Episode 25 finished by Worker-3 with reward 17.0
Episode 26 finished by Worker-2 with reward 22.0
Episode 27 finished by Worker-1 with reward 10.0
Episode 28 finished by Worker-0 with reward 12.0
Episode 29 finished by Worker-2 with reward 38.0
Episode 30 finished by Worker-3 with reward 17.0
Episode 31 finished by Worker-0 with reward 16.0
Episode 32 finished by Worker-1 with reward 13.0
Episode 33 finished by Worker-3 with reward 11.0
Episode 34 finished by Worker-2 with reward 36.0
Episode 35 finished by Worker-0 with reward 14.0
Episode 36 finished by Worker-3 with reward 16.0
Episode 37 finished by Worker-1 with reward 12.0
Episode 38 finished by Worker-0 with reward 36.0
Episode 39 finished by Worker-3 with reward 26.0
Episode 40 finished by Worker-1 with reward 35.0
Episode 41 finished by Worker-0 with reward 15.0
Episode 42 finished by Worker-2 with reward 77.0
Episode 43 finished by Worker-2 with reward 15.0
Episode 44 finished by Worker-1 with reward 40.0
Episode 45 finished by Worker-3 with reward 41.0
Episode 46 finished by Worker-0 with reward 61.0
Episode 47 finished by Worker-2 with reward 26.0
Episode 48 finished by Worker-1 with reward 44.0
Episode 49 finished by Worker-3 with reward 43.0
Episode 50 finished by Worker-0 with reward 26.0
Episode 51 finished by Worker-2 with reward 13.0
Episode 52 finished by Worker-1 with reward 42.0
Episode 53 finished by Worker-3 with reward 16.0
Episode 54 finished by Worker-2 with reward 11.0
Episode 55 finished by Worker-3 with reward 38.0
Episode 56 finished by Worker-1 with reward 18.0
Episode 57 finished by Worker-0 with reward 62.0
Episode 58 finished by Worker-1 with reward 12.0
Episode 59 finished by Worker-2 with reward 13.0
Episode 60 finished by Worker-3 with reward 9.0
Episode 61 finished by Worker-0 with reward 23.0
Episode 62 finished by Worker-2 with reward 12.0
Episode 63 finished by Worker-0 with reward 12.0
Episode 64 finished by Worker-3 with reward 11.0
Episode 65 finished by Worker-1 with reward 33.0
Episode 66 finished by Worker-0 with reward 10.0
Episode 67 finished by Worker-2 with reward 14.0
Episode 68 finished by Worker-1 with reward 13.0
Episode 69 finished by Worker-3 with reward 25.0
Episode 70 finished by Worker-0 with reward 21.0
Episode 71 finished by Worker-1 with reward 19.0
Episode 72 finished by Worker-3 with reward 11.0
Episode 73 finished by Worker-2 with reward 49.0
Episode 74 finished by Worker-0 with reward 23.0
Episode 75 finished by Worker-3 with reward 11.0
Episode 76 finished by Worker-2 with reward 18.0
Episode 77 finished by Worker-1 with reward 44.0
Episode 78 finished by Worker-0 with reward 12.0
Episode 79 finished by Worker-3 with reward 30.0
Episode 80 finished by Worker-1 with reward 20.0
Episode 81 finished by Worker-0 with reward 23.0
Episode 82 finished by Worker-2 with reward 14.0
Episode 83 finished by Worker-1 with reward 11.0
Episode 84 finished by Worker-0 with reward 12.0
Episode 85 finished by Worker-3 with reward 30.0
Episode 86 finished by Worker-2 with reward 83.0
Episode 87 finished by Worker-1 with reward 26.0
Episode 88 finished by Worker-0 with reward 17.0
Episode 89 finished by Worker-3 with reward 17.0
Episode 90 finished by Worker-2 with reward 31.0
Episode 91 finished by Worker-1 with reward 28.0
Episode 92 finished by Worker-0 with reward 12.0
Episode 93 finished by Worker-3 with reward 18.0
Episode 94 finished by Worker-2 with reward 12.0
Episode 95 finished by Worker-3 with reward 17.0
Episode 96 finished by Worker-0 with reward 21.0
Episode 97 finished by Worker-1 with reward 39.0
Episode 98 finished by Worker-2 with reward 44.0
Episode 99 finished by Worker-0 with reward 23.0
Episode 100 finished by Worker-3 with reward 12.0
Episode 101 finished by Worker-1 with reward 14.0
Episode 102 finished by Worker-2 with reward 21.0
Episode 103 finished by Worker-0 with reward 12.0
Episode 104 finished by Worker-3 with reward 18.0
Episode 105 finished by Worker-1 with reward 12.0
Episode 106 finished by Worker-0 with reward 30.0
Episode 107 finished by Worker-2 with reward 21.0
Episode 108 finished by Worker-3 with reward 23.0
Episode 109 finished by Worker-1 with reward 34.0
Episode 110 finished by Worker-2 with reward 13.0
Episode 111 finished by Worker-0 with reward 12.0
Episode 112 finished by Worker-3 with reward 14.0
Episode 113 finished by Worker-1 with reward 21.0
Episode 114 finished by Worker-0 with reward 25.0
Episode 115 finished by Worker-2 with reward 18.0
Episode 116 finished by Worker-3 with reward 23.0
Episode 117 finished by Worker-0 with reward 17.0
Episode 118 finished by Worker-2 with reward 16.0
Episode 119 finished by Worker-1 with reward 24.0
Episode 120 finished by Worker-3 with reward 13.0
Episode 121 finished by Worker-0 with reward 25.0
Episode 122 finished by Worker-1 with reward 14.0
Episode 124 finished by Worker-3 with reward 26.0
Episode 123 finished by Worker-2 with reward 13.0
Episode 125 finished by Worker-0 with reward 20.0
Episode 126 finished by Worker-2 with reward 12.0
Episode 127 finished by Worker-1 with reward 15.0
Episode 128 finished by Worker-3 with reward 9.0
Episode 129 finished by Worker-1 with reward 14.0
Episode 130 finished by Worker-0 with reward 15.0
Episode 131 finished by Worker-2 with reward 14.0
Episode 132 finished by Worker-3 with reward 18.0
Episode 133 finished by Worker-2 with reward 31.0
Episode 134 finished by Worker-3 with reward 30.0
Episode 135 finished by Worker-0 with reward 14.0
Episode 136 finished by Worker-1 with reward 16.0
Episode 137 finished by Worker-3 with reward 14.0
Episode 138 finished by Worker-0 with reward 15.0
Episode 140 finished by Worker-2 with reward 25.0
Episode 139 finished by Worker-1 with reward 20.0
Episode 141 finished by Worker-3 with reward 15.0
Episode 142 finished by Worker-0 with reward 15.0
Episode 143 finished by Worker-2 with reward 11.0
Episode 144 finished by Worker-1 with reward 14.0
Episode 145 finished by Worker-3 with reward 17.0
Episode 146 finished by Worker-2 with reward 18.0
Episode 147 finished by Worker-3 with reward 11.0
Episode 148 finished by Worker-1 with reward 26.0
Episode 149 finished by Worker-0 with reward 20.0
Episode 150 finished by Worker-3 with reward 14.0
Episode 151 finished by Worker-2 with reward 20.0
Episode 152 finished by Worker-1 with reward 19.0
Episode 153 finished by Worker-0 with reward 22.0
Episode 154 finished by Worker-2 with reward 15.0
Episode 155 finished by Worker-3 with reward 15.0
Episode 156 finished by Worker-1 with reward 31.0
Episode 157 finished by Worker-0 with reward 18.0
Episode 158 finished by Worker-2 with reward 66.0
Episode 159 finished by Worker-3 with reward 24.0
Episode 160 finished by Worker-1 with reward 25.0
Episode 161 finished by Worker-0 with reward 17.0
Episode 162 finished by Worker-2 with reward 30.0
Episode 163 finished by Worker-3 with reward 20.0
Episode 164 finished by Worker-1 with reward 10.0
Episode 165 finished by Worker-0 with reward 16.0
Episode 166 finished by Worker-2 with reward 13.0
Episode 167 finished by Worker-3 with reward 27.0
Episode 168 finished by Worker-0 with reward 15.0
Episode 169 finished by Worker-1 with reward 11.0
Episode 170 finished by Worker-2 with reward 17.0
Episode 171 finished by Worker-0 with reward 21.0
Episode 172 finished by Worker-3 with reward 45.0
Episode 173 finished by Worker-1 with reward 34.0
Episode 174 finished by Worker-3 with reward 33.0
Episode 175 finished by Worker-2 with reward 22.0
Episode 176 finished by Worker-0 with reward 15.0
Episode 177 finished by Worker-1 with reward 14.0
Episode 178 finished by Worker-2 with reward 15.0
Episode 179 finished by Worker-3 with reward 30.0
Episode 180 finished by Worker-1 with reward 12.0
Episode 181 finished by Worker-2 with reward 17.0
Episode 182 finished by Worker-3 with reward 9.0
Episode 183 finished by Worker-0 with reward 57.0
Episode 184 finished by Worker-0 with reward 11.0
Episode 185 finished by Worker-1 with reward 15.0
Episode 186 finished by Worker-3 with reward 25.0
Episode 187 finished by Worker-0 with reward 23.0
Episode 188 finished by Worker-2 with reward 22.0
Episode 189 finished by Worker-1 with reward 35.0
Episode 190 finished by Worker-3 with reward 20.0
Episode 191 finished by Worker-0 with reward 15.0
Episode 192 finished by Worker-2 with reward 32.0
Episode 193 finished by Worker-1 with reward 11.0
Episode 194 finished by Worker-3 with reward 11.0
Episode 195 finished by Worker-0 with reward 19.0
Episode 196 finished by Worker-2 with reward 23.0
Episode 197 finished by Worker-1 with reward 14.0
Episode 198 finished by Worker-3 with reward 13.0
Episode 199 finished by Worker-0 with reward 23.0
Episode 200 finished by Worker-2 with reward 38.0
Episode 201 finished by Worker-1 with reward 23.0
Episode 202 finished by Worker-3 with reward 20.0
Episode 203 finished by Worker-0 with reward 17.0
Episode 204 finished by Worker-1 with reward 15.0
Episode 205 finished by Worker-2 with reward 47.0
Episode 206 finished by Worker-0 with reward 17.0
Episode 207 finished by Worker-3 with reward 16.0
Episode 208 finished by Worker-1 with reward 28.0
Episode 209 finished by Worker-2 with reward 18.0
Episode 210 finished by Worker-0 with reward 17.0
Episode 211 finished by Worker-3 with reward 16.0
Episode 212 finished by Worker-2 with reward 29.0
Episode 213 finished by Worker-1 with reward 20.0
Episode 214 finished by Worker-0 with reward 47.0
Episode 215 finished by Worker-3 with reward 18.0
Episode 216 finished by Worker-1 with reward 28.0
Episode 217 finished by Worker-2 with reward 17.0
Episode 218 finished by Worker-0 with reward 12.0
Episode 219 finished by Worker-3 with reward 47.0
Episode 220 finished by Worker-2 with reward 18.0
Episode 221 finished by Worker-1 with reward 16.0
Episode 222 finished by Worker-0 with reward 10.0
Episode 223 finished by Worker-3 with reward 11.0
Episode 224 finished by Worker-2 with reward 15.0
Episode 225 finished by Worker-1 with reward 21.0
Episode 226 finished by Worker-3 with reward 30.0
Episode 227 finished by Worker-0 with reward 22.0
Episode 228 finished by Worker-2 with reward 39.0
Episode 229 finished by Worker-1 with reward 65.0
Episode 230 finished by Worker-3 with reward 29.0
Episode 231 finished by Worker-0 with reward 38.0
Episode 232 finished by Worker-2 with reward 46.0
Episode 233 finished by Worker-3 with reward 14.0
Episode 234 finished by Worker-1 with reward 16.0
Episode 235 finished by Worker-0 with reward 10.0
Episode 236 finished by Worker-2 with reward 13.0
Episode 237 finished by Worker-3 with reward 9.0
Episode 238 finished by Worker-1 with reward 10.0
Episode 239 finished by Worker-0 with reward 25.0
Episode 240 finished by Worker-3 with reward 16.0
Episode 241 finished by Worker-2 with reward 39.0
Episode 242 finished by Worker-1 with reward 22.0
Episode 243 finished by Worker-2 with reward 11.0
Episode 244 finished by Worker-0 with reward 28.0
Episode 245 finished by Worker-3 with reward 10.0
Episode 246 finished by Worker-1 with reward 26.0
Episode 247 finished by Worker-0 with reward 13.0
Episode 248 finished by Worker-2 with reward 20.0
Episode 249 finished by Worker-3 with reward 31.0
Episode 250 finished by Worker-1 with reward 11.0
Episode 251 finished by Worker-3 with reward 18.0
Episode 252 finished by Worker-2 with reward 28.0
Episode 253 finished by Worker-0 with reward 75.0
Episode 254 finished by Worker-1 with reward 79.0
Episode 255 finished by Worker-3 with reward 12.0
Episode 256 finished by Worker-2 with reward 57.0
Episode 257 finished by Worker-0 with reward 80.0
Episode 258 finished by Worker-1 with reward 61.0
Episode 259 finished by Worker-3 with reward 17.0
Episode 260 finished by Worker-0 with reward 9.0
Episode 261 finished by Worker-2 with reward 15.0
Episode 262 finished by Worker-1 with reward 15.0
Episode 263 finished by Worker-3 with reward 26.0
Episode 264 finished by Worker-0 with reward 17.0
Episode 265 finished by Worker-1 with reward 20.0
Episode 266 finished by Worker-2 with reward 11.0
Episode 267 finished by Worker-3 with reward 28.0
Episode 268 finished by Worker-0 with reward 12.0
Episode 269 finished by Worker-2 with reward 13.0
Episode 270 finished by Worker-3 with reward 37.0
Episode 271 finished by Worker-1 with reward 40.0
Episode 272 finished by Worker-0 with reward 11.0
Episode 273 finished by Worker-2 with reward 21.0
Episode 274 finished by Worker-3 with reward 19.0
Episode 275 finished by Worker-1 with reward 42.0
Episode 276 finished by Worker-2 with reward 18.0
Episode 277 finished by Worker-3 with reward 11.0
Episode 278 finished by Worker-0 with reward 97.0
Episode 279 finished by Worker-1 with reward 10.0
Episode 280 finished by Worker-2 with reward 11.0
Episode 281 finished by Worker-0 with reward 13.0
Episode 282 finished by Worker-3 with reward 15.0
Episode 283 finished by Worker-1 with reward 15.0
Episode 284 finished by Worker-2 with reward 36.0
Episode 285 finished by Worker-3 with reward 14.0
Episode 286 finished by Worker-0 with reward 14.0
Episode 287 finished by Worker-1 with reward 14.0
Episode 288 finished by Worker-2 with reward 11.0
Episode 289 finished by Worker-1 with reward 21.0
Episode 290 finished by Worker-0 with reward 41.0
Episode 291 finished by Worker-3 with reward 13.0
Episode 292 finished by Worker-2 with reward 12.0
Episode 293 finished by Worker-1 with reward 9.0
Episode 294 finished by Worker-2 with reward 13.0
Episode 295 finished by Worker-0 with reward 12.0
Episode 296 finished by Worker-3 with reward 50.0
Episode 297 finished by Worker-1 with reward 65.0
Episode 298 finished by Worker-2 with reward 56.0
Episode 299 finished by Worker-0 with reward 25.0
Episode 300 finished by Worker-3 with reward 18.0
Episode 301 finished by Worker-1 with reward 17.0
Episode 302 finished by Worker-2 with reward 15.0
Episode 303 finished by Worker-0 with reward 30.0

六、总结与扩展

本文实现了 A3C 算法的核心逻辑，展示了并行化训练在强化学习中的优势。读者可以尝试以下扩展：

  1. 在 Atari 游戏等复杂环境中测试算法

  2. 调整 Worker 数量观察训练效率变化

  3. 实现梯度裁剪等稳定化技巧

在下一篇文章中，我们将探讨分布式强化学习的进阶技术——IMPALA 算法，敬请期待！

注意事项：

  1. 本代码需在支持多进程的环境下运行（如 Linux）

  2. 可添加 torch.nn.utils.clip_grad_norm_ 防止梯度爆炸

  3. 使用 gym.wrappers.Monitor 可录制智能体行为

希望本文能帮助您理解 A3C 算法的核心思想与实践方法！欢迎在评论区交流讨论。

相关文章：

PyTorch 深度学习实战（17）：Asynchronous Advantage Actor-Critic (A3C) 算法与并行训练

在上一篇文章中&#xff0c;我们深入探讨了 Soft Actor-Critic (SAC) 算法及其在平衡探索与利用方面的优势。本文将介绍强化学习领域的重要里程碑——Asynchronous Advantage Actor-Critic (A3C) 算法&#xff0c;并展示如何利用 PyTorch 实现并行化训练来加速学习过程。 一、A…...

编程日记 2026/2/16 7:55:40

【服务器知识】Nginx路由匹配规则说明

Nginx路由匹配规则说明 **一、Nginx路由匹配核心机制****二、匹配规则语法详解**1. **精确匹配 ()**2. **前缀匹配 (^~ 或 /)**3. **正则匹配 (~ 或 ~*)**4. **通配符匹配 (*)** **三、路由匹配优先级顺序****四、高级路由技巧**1. **条件判断 (if语句)**2. **路径重写 (rewrit…...

编程日记 2025/6/24 11:15:58

Docker换源加速(更换镜像源)详细教程（2025.3最新可用镜像，全网最详细）

文章目录 前言可用镜像源汇总换源方法1-临时换源换源方法2-永久换源&#xff08;推荐&#xff09;常见问题及对应解决方案1.换源后&#xff0c;可以成功pull&#xff0c;但是search会出错 补充1.如何测试镜像源是否可用2.Docker内的Linux换源教程 换源速通版&#xff08;可以直…...

编程日记 2026/2/16 1:18:22

OpenAI--Agent SDK简介

项目概述 OpenAI Agents SDK 是一个轻量级但功能强大的框架&#xff0c;用于构建多智能体工作流。它主要利用大语言模型&#xff08;LLM&#xff09;&#xff0c;通过配置智能体、交接、护栏和跟踪等功能&#xff0c;实现复杂的工作流管理。以下是对其各个部分运行过程和代码流…...

编程日记 2026/3/1 9:18:14

pytorch小记（十三）：pytorch中`nn.ModuleList` 详解

pytorch小记&#xff08;十三&#xff09;&#xff1a;pytorch中nn.ModuleList 详解 PyTorch 中的 nn.ModuleList 详解1. 什么是 nn.ModuleList&#xff1f;2. 为什么不直接使用普通的 Python 列表&#xff1f;3. nn.ModuleList 的基本用法示例&#xff1a;构建一个包含两层全连…...

编程日记 2026/2/16 12:32:13

SpringData Redis：RedisTemplate配置与数据操作

文章目录 引言一、Redis概述与环境准备二、RedisTemplate基础配置三、连接属性配置四、操作String类型数据五、操作Hash类型数据六、操作List类型数据七、操作Set类型数据八、操作ZSet类型数据九、事务与管道操作总结 引言 Redis作为高性能的NoSQL数据库&#xff0c;在分布式系…...

编程日记 2026/2/14 14:06:25

Qt按钮控件常用的API

1.创建按钮 QPushButton *btnnew QPushButton; 以顶层方式弹出窗口控件 代码&#xff1a; #include "widget.h" #include "ui_widget.h" #include"QPushButton"Widget::Widget(QWidget *parent): QWidget(parent), ui(new Ui::Widget) {ui-&…...

编程日记 2026/2/19 20:44:05

如何检查CMS建站系统的插件是否安全？

检查好CMS建站系统的插件安全是确保网站安全的重要环节&#xff0c;对于常见的安全检查&#xff0c;大家可以利用以下几种有效的方法和工具&#xff0c;来帮你评估插件的安全性。 1. 检查插件来源和开发者信誉 选择可信来源&#xff1a;仅从官方插件库或可信的第三方开发者处…...

编程日记 2026/2/23 2:40:46

修改HuggingFace模型默认缓存路径

huggingface模型的默认缓存路径是~/.cache/huggingface/hub/ 通常修改为自己的路径会更为方便。 方式一&#xff1a;cache_dir 参数 可以通过from_pretrained函数中的 cache_dir 参数来指定&#xff0c;缺点&#xff0c;每次都需要手动指定&#xff0c;比较麻烦。 如&#x…...

编程日记 2026/2/18 12:53:19

ORA-12541: TNS:no listener

问题描述&#xff1a;使用 PL/SQL Developer 连接数据库时出现ORA-12541错误。 1.查询监听状态 [oraclelocalhost admin]$ lsnrctl statusLSNRCTL for Linux: Version 19.0.0.0.0 - Production on 15-MAR-2025 10:47:57Copyright (c) 1991, 2021, Oracle. All rights reserv…...

编程日记 2026/2/14 17:53:19

【Matlab GUI】封装matlab GUI为exe文件

注&#xff1a;封装后的exe还是需要有matlab环境才能运行 &#xff08;1&#xff09;安装MCRinstaller.exe文件&#xff0c;在matlab安装目录下的toolbox/compiler/deploy/win64文件夹里 &#xff08;2&#xff09;安装完MCRinstaller.exe&#xff0c;字命令窗口输入&#x…...

编程日记 2026/2/16 2:53:09

【eNSP实战】（续）一个AC多个VAP的实现—将隧道转发改成直接转发

在 一个AC多个VAP的实现—CAPWAP隧道转发 此篇文章配置的基础上&#xff0c;将隧道转发改成直接转发 一、改成直接转发需要改动的配置 &#xff08;一&#xff09;将连接AP的接口改成trunk口&#xff0c;并允许vlan100、101、102通过 [AC1]interface GigabitEthernet 0/0/8 …...

编程日记 2026/2/18 19:55:10

Redis能否替代MySQL作为主数据库？深入解析两者的持久化差异与适用边界——基于AOF持久化与关系型数据库的对比

一、Redis的持久化机制与可靠性分析 ​AOF持久化原理与策略 Redis的AOF&#xff08;Append Only File&#xff09;通过记录所有写操作命令实现持久化&#xff0c;支持三种策略&#xff1a; ​**always模式**&#xff1a;每条命令执行后立即同步到磁盘&#xff0c;理论上数据丢失…...

编程日记 2026/2/16 0:24:54

机器人ROS学习：Ubuntu22.04安装ROS2和Moveit2实现运动规划

通过本篇文章学习&#xff0c;你可以收获以下内容&#xff1a; 学会在 Ubuntu22.04 上安装 Moveit2学会下载编译运行 Moveit2 样例程序学会使用样例程序进行运动规划等 版本平台 系统版本&#xff1a;ubuntu22.04ROS2 版本&#xff1a;humbleMoveit 版本&#xff1a;moveit2…...

编程日记 2026/3/1 12:41:01

GIT标签（Tag）操作

在Git中&#xff0c;标签&#xff08;Tag&#xff09;用于标记特定的提交点&#xff0c;通常用于发布版本。 切换到需要打标签的分支&#xff1a; git checkout <branch-name>创建标签 git tag v1.0.0 git tag -a v1.0.0 -m "Release version 1.0.0"查看所…...

编程日记 2026/2/15 0:32:04

生成式AI红队测试：如何有效评估大语言模型

OWASP最新指南为组建生成式AI红队或调整现有红队以适应新技术提供了详细的指导。 红队测试是一种经过时间检验的网络安全系统测试和加固方法&#xff0c;但它需要不断适应技术的演变。近年来&#xff0c;生成式AI和大语言模型&#xff08;LLM&#xff09;的爆发&#xff0c;是…...

编程日记 2026/2/19 16:16:48

技术路线图ppt模板_流程图ppt图表_PPT架构图

技术路线图ppt模板 / 学术ppt模板 - 院士增选、国家科技奖、杰青、长江学者特聘教授、校企联聘教授、重点研发、优青、青长、青拔.. / 学术ppt案例 WordinPPT / 持续为双一流高校、科研院所、企业等提供PPT制作系统服务。 - 科学技术奖ppt&#xff1a;自然科学奖 | 技术…...

编程日记 2026/2/26 16:30:03

Leetcode-131.Palindrome Partitioning [C++][Java]

目录 一、题目描述 二、解题思路 【C】 【Java】 Leetcode-131.Palindrome Partitioninghttps://leetcode.com/problems/palindrome-partitioning/description/131. 分割回文串 - 力扣&#xff08;LeetCode&#xff09;131. 分割回文串 - 给你一个字符串 s&#xff0c;请你…...

编程日记 2026/2/21 17:15:07

LeetCode 解题思路 20（Hot 100）

解题思路&#xff1a; 递归定义对称性&#xff1a; 若两棵子树镜像对称&#xff0c;需满足&#xff1a; 当前节点值相等&#xff1b;左子树的左节点与右子树的右节点对称&#xff1b;左子树的右节点与右子树的左节点对称。 终止条件&#xff1a; 两个节点均为空 → 对称&am…...

编程日记 2026/2/24 9:55:28

挖矿------获取以太坊测试币

文章目录 挖矿------获取以太坊测试币通过水龙头获取以太坊测试币了解Sepolia是什么&#xff1f;水龙头&#xff08;Faucet&#xff09;是什么&#xff1f;Gitcoin Passport是什么&#xff1f; 操作1.MetaMask钱包2.将MetaMask切换到Sepolia测试网络3.用MetaMask连接Gitcoin Pa…...

编程日记 2026/2/28 6:25:22

每天五分钟深度学习框架pytorch:基于pytorch搭建循环神经网络RNN

本文重点 我们前面介绍了循环神经网络RNN,主要分析了它的维度信息，其实它的维度信息是最重要的，一旦我们把维度弄清楚了，一起就很简单了，本文我们正式的来学习一下，如何使用pytorch搭建循环神经网络RNN。 RNN的搭建 在pytorch中我们使用nn.RNN()就可以创建出RNN神经网络…...

编程日记 2026/2/14 18:57:42

XEasyWork：面向AI应用的可视化工作流开发平台

文章目录 前言 一、平台核心价值 1.1产品定位 1.2 技术优势 二、技术架构解析 2.1战略级整合 自主开发模块 2.2集成开源项目 三、体验地址 三、未来规划 总结 前言 在人工智能技术快速落地的今天&#xff0c;开发者在构建AI应用时仍面临两大挑战&#xff1a;技术栈复杂带来的高…...

编程日记 2026/2/25 3:43:41

C#进阶（多线程相关）

1。进程&#xff1f; 进程&#xff08;Process&#xff09;是计算机中的程序关于某数据集合上的一次运行活动&#xff0c;【是系统进行资源分配的基本单位】&#xff0c;是操作系统结构的基础。在早期面向进程设计的计算机结构中&#xff0c;进程是程序的基本执行实体&#xf…...

编程日记 2026/2/15 16:47:51

【C++】：C++11详解 —— 右值引用

目录 左值和右值 左值的概念 右值的概念 左值 vs 右值 左值引用 和 右值引用 左值引用 右值引用 左值引用 vs 右值引用 使用场景 左值引用的使用场景 左值引用的短板 右值引用的使用场景 1. 实现移动语义&#xff08;资源高效转移&#xff09; 2. 优化容器操作&a…...

编程日记 2026/2/27 15:47:10

【css酷炫效果】纯CSS实现虫洞穿越效果

【css酷炫效果】纯CSS实现穿越效果 缘创作背景html结构css样式完整代码基础版进阶版&#xff08;虫洞穿越&#xff09; 效果图 想直接拿走的老板&#xff0c;链接放在这里&#xff1a;https://download.csdn.net/download/u011561335/90491973 缘 创作随缘&#xff0c;不定时…...

编程日记 2026/2/23 8:58:38

Linux IP 配置

Linux IP 配置 1 环境介绍2 网卡信息配置3 使用nmtui工具配置4 更多Linux命令学习使用列表 1 环境介绍 虚拟机&#xff0c;服务器安装系统完成后&#xff0c;先要配置ip 地址&#xff0c;这样可以方便远程若是物理服务器一般会有4个网卡信息麒麟v10&#xff0c;CentOS7&#x…...

编程日记 2026/3/1 5:06:51

设计模式之装饰器模式：原理、实现与应用

引言 装饰器模式&#xff08;Decorator Pattern&#xff09;是一种结构型设计模式&#xff0c;它允许你通过将对象放入包含行为的特殊封装对象中来为原对象动态添加新的行为。装饰器模式提供了一种灵活的替代方案&#xff0c;避免了通过继承扩展功能的局限性。本文将深入探讨装…...

编程日记 2026/2/14 19:51:05

c语言笔记 结构体基础

目录 基础知识 结构体定义 基础知识 在c语言中变量是有类型的&#xff0c;比如整型&#xff0c;char型&#xff0c;浮点型等&#xff0c;这些都是单一的类型&#xff0c;那么如果说我要定义一个学生的信息&#xff0c;那么这些单一的类型是不足以表达一个学生的全部信息&#…...

编程日记 2025/12/14 16:10:19

Vue 登录 记住密码，设置存储时间

Vue 登录 记住密码&#xff0c;设置存储时间 一、手动存储login.vue 二、使用vue-cookies插件main.jslogin.vue 一、手动存储 login.vue 提示&#xff1a; // 设置cookie方法 setCookie(loginName, password, days) {let text encryptDes(password, des123)//使用des方法加…...

编程日记 2025/12/11 17:54:36

C++ 学习笔记（三）—— 入门+类和对象

1、内联函数&#xff08;inline&#xff09; 内联函数主要是解决C语言的宏的缺陷提出来的&#xff1b; 宏的缺陷&#xff1a; 1&#xff09;容易出错&#xff0c;语法坑很多&#xff1b; 2&#xff09;不能调试&#xff1b; 3&#xff09;没有类型安全的检查&#xff1b; 宏的…...

编程日记 2026/2/14 1:30:14