Azure Machine Learning - Azure OpenAI GPT 3.5 Turbo 微调教程
本教程将引导你在Azure平台完成对
gpt-35-turbo-0613
模型的微调。
关注TechLead,分享AI全维度知识。作者拥有10+年互联网服务架构、AI产品研发经验、团队管理经验,同济本复旦硕,复旦机器人智能实验室成员,阿里云认证的资深架构师,项目管理专业人士,上亿营收AI产品研发负责人
教程介绍
本教程介绍如何执行下列操作:
- 创建示例微调数据集。
- 为资源终结点和 API 密钥创建环境变量。
- 准备样本训练和验证数据集以进行微调。
- 上传训练文件和验证文件进行微调。
- 为
gpt-35-turbo-0613
创建微调作业。 - 部署自定义微调模型。
环境准备
-
Azure 订阅 - 免费创建订阅。
-
已在所需的 Azure 订阅中授予对 Azure OpenAI 的访问权限 目前,仅应用程序授予对此服务的访问权限。 可以通过在 https://aka.ms/oai/access 上填写表单来申请对 Azure OpenAI 的访问权限。
-
Python 3.7.1 或更高版本
-
以下 Python 库:
json
、requests
、os
、tiktoken
、time
、openai
。 -
OpenAI Python 库应至少为版本
1.0
。 -
Jupyter Notebook
-
[可进行
gpt-35-turbo-0613
微调的区域]中的 Azure OpenAI 资源。 -
微调访问需要认知服务 OpenAI 参与者。
设置
Python 库
- OpenAI Python 1.x
pip install openai json requests os tiktoken time
检索密钥和终结点
若要成功对 Azure OpenAI 发出调用,需要一个终结点和一个密钥。
变量名称 | 值 |
---|---|
ENDPOINT | 从 Azure 门户检查资源时,可在“密钥和终结点”部分中找到此值。 或者,可以在“Azure OpenAI Studio”>“操场”>“代码视图”中找到该值。 示例终结点为:https://docs-test-001.openai.azure.com/ 。 |
API-KEY | 从 Azure 门户检查资源时,可在“密钥和终结点”部分中找到此值。 可以使用 KEY1 或 KEY2 。 |
在 Azure 门户中转到你的资源。 可以在“资源管理”部分找到“终结点和密钥”。 复制终结点和访问密钥,因为在对 API 调用进行身份验证时需要这两项。 可以使用 KEY1
或 KEY2
。 始终准备好两个密钥可以安全地轮换和重新生成密钥,而不会导致服务中断。
环境变量
- 命令行
setx AZURE_OPENAI_API_KEY "REPLACE_WITH_YOUR_KEY_VALUE_HERE"
setx AZURE_OPENAI_ENDPOINT "REPLACE_WITH_YOUR_ENDPOINT_HERE"
- PowerShell
[System.Environment]::SetEnvironmentVariable('AZURE_OPENAI_API_KEY', 'REPLACE_WITH_YOUR_KEY_VALUE_HERE', 'User')
[System.Environment]::SetEnvironmentVariable('AZURE_OPENAI_ENDPOINT', 'REPLACE_WITH_YOUR_ENDPOINT_HERE', 'User')
- Bash
echo export AZURE_OPENAI_API_KEY="REPLACE_WITH_YOUR_KEY_VALUE_HERE" >> /etc/environment
echo export AZURE_OPENAI_ENDPOINT="REPLACE_WITH_YOUR_ENDPOINT_HERE" >> /etc/environmentsource /etc/environment
创建示例数据集
微调 gpt-35-turbo-0613
需要特殊格式的 JSONL 训练文件。 OpenAI 在其文档中提供了以下示例:
{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the capital of France?"}, {"role": "assistant", "content": "Paris, as if everyone doesn't know that already."}]}
{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who wrote 'Romeo and Juliet'?"}, {"role": "assistant", "content": "Oh, just some guy named William Shakespeare. Ever heard of him?"}]}
{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "How far is the Moon from Earth?"}, {"role": "assistant", "content": "Around 384,400 kilometers. Give or take a few, like that really matters."}]}
对于本示例,我们将稍作修改,将其更改为:
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the capital of France?"}, {"role": "assistant", "content": "Paris, as if everyone doesn't know that already."}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who wrote 'Romeo and Juliet'?"}, {"role": "assistant", "content": "Oh, just some guy named William Shakespeare. Ever heard of him?"}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "How far is the Moon from Earth?"}, {"role": "assistant", "content": "Around 384,400 kilometers. Give or take a few, like that really matters."}]}
尽管这三个示例有助于提供常规格式,但如果你想引导自定义微调模型以类似的方式进行响应,则需要更多示例。 一般来说,你需要至少 50 个高质量示例来开始。然而,你完全有可能有一个用例,它可能需要 1,000 个高质量的训练示例才能成功。
通常,将数据集大小加倍会促使模型质量呈线性增长。 但请记住,低质量的示例可能会对性能产生负面影响。 如果你根据大量内部数据来训练模型,而不是先修剪数据集以便获得最高质量的示例,则最终可能会得到一个性能比预期差得多的模型。
需要创建两个文件:training_set.jsonl
和 validation_set.jsonl
。
在运行 Jupyter Notebook 的同一目录中创建文件,并将以下代码块的内容复制到相应的文件中:
training_set.jsonl
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who discovered Antarctica?"}, {"role": "assistant", "content": "Some chaps named Fabian Gottlieb von Bellingshausen and Mikhail Lazarev, as if they don't teach that in every school!"}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What is the biggest ocean?"}, {"role": "assistant", "content": "The Pacific Ocean. It's not like it's a small pond or anything."}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What is the largest planet?"}, {"role": "assistant", "content": "It's called Jupiter, you may have heard of it...or not."}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who painted the Mona Lisa?"}, {"role": "assistant", "content": "Oh, some obscure artist named Leonardo da Vinci. Not that he is famous or anything."}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the boiling point of water?"}, {"role": "assistant", "content": "Exciting fact of the day: it's 100 degrees Celsius. Life-changing, isn't it?"}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who wrote 'The Odyssey'?"}, {"role": "assistant", "content": "Oh, just some poet named Homer. Nothing special, really."}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What is the largest mammal?"}, {"role": "assistant", "content": "Blue Whale. Quite a big fish isn't it? Oh wait, it's not a fish!"}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the highest mountain peak?"}, {"role": "assistant", "content": "Everest. Brace yourself, it's not a walk in the park."}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who invented the telephone?"}, {"role": "assistant", "content": "A guy named Alexander Graham Bell. Bet you didn't guess that one!"}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What is the distance from Earth to the Sun?"}, {"role": "assistant", "content": "About 93 million miles. Just a quick drive, really."}]}
validation_set.jsonl
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the capital of Australia?"}, {"role": "assistant", "content": "It's Canberra, not Sydney. Shocking, I know!"}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who wrote 'The Great Gatsby'?"}, {"role": "assistant", "content": "Some guy named F. Scott Fitzgerald. Ever heard of him?"}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the deepest part of the ocean?"}, {"role": "assistant", "content": "Mariana Trench, makes your swimming pool seem a bit shallow, doesn't it?"}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who composed the 'Moonlight Sonata'?"}, {"role": "assistant", "content": "Just a little-known guy named Beethoven. No big deal."}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What are the colors of the rainbow?"}, {"role": "assistant", "content": "ROYGBIV. Red, orange, yellow, green, blue, indigo, violet. Not that we learned this in preschool or anything!"}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What is the smallest planet?"}, {"role": "assistant", "content": "Meet Mercury, the runt of the solar system family."}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who wrote 'Pride and Prejudice'?"}, {"role": "assistant", "content": "Do the words Jane Austen ring a bell? No? Well, she wrote it."}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the largest desert?"}, {"role": "assistant", "content": "Antarctica. Surprise, surprise! Deserts aren't just full of sand, you know."}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What is the longest river?"}, {"role": "assistant", "content": "The Nile River. It's not like it's famous or anything."}]}
{"messages": [{"role": "system", "content": "Clippy is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the capital of Germany?"}, {"role": "assistant", "content": "Berlin. Shocking news, right?"}]}
现在,你需要对训练和验证文件运行一些初步检查。
import json# Load the training set
with open('training_set.jsonl', 'r', encoding='utf-8') as f:training_dataset = [json.loads(line) for line in f]# Training dataset stats
print("Number of examples in training set:", len(training_dataset))
print("First example in training set:")
for message in training_dataset[0]["messages"]:print(message)# Load the validation set
with open('validation_set.jsonl', 'r', encoding='utf-8') as f:validation_dataset = [json.loads(line) for line in f]# Validation dataset stats
print("\nNumber of examples in validation set:", len(validation_dataset))
print("First example in validation set:")
for message in validation_dataset[0]["messages"]:print(message)
输出:
Number of examples in training set: 10
First example in training set:
{'role': 'system', 'content': 'Clippy is a factual chatbot that is also sarcastic.'}
{'role': 'user', 'content': 'Who discovered America?'}
{'role': 'assistant', 'content': "Some chap named Christopher Columbus, as if they don't teach that in every school!"}Number of examples in validation set: 10
First example in validation set:
{'role': 'system', 'content': 'Clippy is a factual chatbot that is also sarcastic.'}
{'role': 'user', 'content': "What's the capital of Australia?"}
{'role': 'assistant', 'content': "It's Canberra, not Sydney. Shocking, I know!"}
在本例中,我们只有 10 个训练示例和 10 个验证示例,因此虽然这将演示微调模型的基本机制,但示例数量不太可能足以产生持续明显的影响。
现在,可以使用 tiktoken 库从 OpenAI 运行一些额外的代码来验证令牌计数。 各个示例需要保持在 gpt-35-turbo-0613
模型的 4096 个令牌的输入令牌限制内。
import json
import tiktoken
import numpy as np
from collections import defaultdictencoding = tiktoken.get_encoding("cl100k_base") # default encoding used by gpt-4, turbo, and text-embedding-ada-002 modelsdef num_tokens_from_messages(messages, tokens_per_message=3, tokens_per_name=1):num_tokens = 0for message in messages:num_tokens += tokens_per_messagefor key, value in message.items():num_tokens += len(encoding.encode(value))if key == "name":num_tokens += tokens_per_namenum_tokens += 3return num_tokensdef num_assistant_tokens_from_messages(messages):num_tokens = 0for message in messages:if message["role"] == "assistant":num_tokens += len(encoding.encode(message["content"]))return num_tokensdef print_distribution(values, name):print(f"\n#### Distribution of {name}:")print(f"min / max: {min(values)}, {max(values)}")print(f"mean / median: {np.mean(values)}, {np.median(values)}")print(f"p5 / p95: {np.quantile(values, 0.1)}, {np.quantile(values, 0.9)}")files = ['training_set.jsonl', 'validation_set.jsonl']for file in files:print(f"Processing file: {file}")with open(file, 'r', encoding='utf-8') as f:dataset = [json.loads(line) for line in f]total_tokens = []assistant_tokens = []for ex in dataset:messages = ex.get("messages", {})total_tokens.append(num_tokens_from_messages(messages))assistant_tokens.append(num_assistant_tokens_from_messages(messages))print_distribution(total_tokens, "total tokens")print_distribution(assistant_tokens, "assistant tokens")print('*' * 50)
输出:
Processing file: training_set.jsonl#### Distribution of total tokens:
min / max: 47, 57
mean / median: 50.8, 50.0
p5 / p95: 47.9, 55.2#### Distribution of assistant tokens:
min / max: 13, 21
mean / median: 16.3, 15.5
p5 / p95: 13.0, 20.1
**************************************************
Processing file: validation_set.jsonl#### Distribution of total tokens:
min / max: 43, 65
mean / median: 51.4, 49.0
p5 / p95: 45.7, 56.9#### Distribution of assistant tokens:
min / max: 8, 29
mean / median: 15.9, 13.5
p5 / p95: 11.6, 20.9
**************************************************
上传微调文件
- OpenAI Python 1.x
# Upload fine-tuning filesimport os
from openai import AzureOpenAIclient = AzureOpenAI(azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), api_key=os.getenv("AZURE_OPENAI_KEY"), api_version="2023-12-01-preview" # This API version or later is required to access fine-tuning for turbo/babbage-002/davinci-002
)training_file_name = 'training_set.jsonl'
validation_file_name = 'validation_set.jsonl'# Upload the training and validation dataset files to Azure OpenAI with the SDK.training_response = client.files.create(file=open(training_file_name, "rb"), purpose="fine-tune"
)
training_file_id = training_response.idvalidation_response = client.files.create(file=open(validation_file_name, "rb"), purpose="fine-tune"
)
validation_file_id = validation_response.idprint("Training file ID:", training_file_id)
print("Validation file ID:", validation_file_id)
输出:
Training file ID: file-9ace76cb11f54fdd8358af27abf4a3ea
Validation file ID: file-70a3f525ed774e78a77994d7a1698c4b
开始微调
现在微调文件已成功上传,可以提交微调训练作业:
- OpenAI Python 1.x
response = client.fine_tuning.jobs.create(training_file=training_file_id,validation_file=validation_file_id,model="gpt-35-turbo-0613", # Enter base model name. Note that in Azure OpenAI the model name contains dashes and cannot contain dot/period characters.
)job_id = response.id# You can use the job ID to monitor the status of the fine-tuning job.
# The fine-tuning job will take some time to start and complete.print("Job ID:", response.id)
print("Status:", response.id)
print(response.model_dump_json(indent=2))
输出:
Job ID: ftjob-40e78bc022034229a6e3a222c927651c
Status: pending
{"hyperparameters": {"n_epochs": 2},"status": "pending","model": "gpt-35-turbo-0613","training_file": "file-90ac5d43102f4d42a3477fd30053c758","validation_file": "file-e21aad7dddbc4ddc98ba35c790a016e5","id": "ftjob-40e78bc022034229a6e3a222c927651c","created_at": 1697156464,"updated_at": 1697156464,"object": "fine_tuning.job"
}
跟踪训练作业状态
如果想轮询训练作业状态,直至其完成,可以运行:
- OpenAI Python 1.x
# Track training statusfrom IPython.display import clear_output
import timestart_time = time.time()# Get the status of our fine-tuning job.
response = client.fine_tuning.jobs.retrieve(job_id)status = response.status# If the job isn't done yet, poll it every 10 seconds.
while status not in ["succeeded", "failed"]:time.sleep(10)response = client.fine_tuning.jobs.retrieve(job_id)print(response.model_dump_json(indent=2))print("Elapsed time: {} minutes {} seconds".format(int((time.time() - start_time) // 60), int((time.time() - start_time) % 60)))status = response.statusprint(f'Status: {status}')clear_output(wait=True)print(f'Fine-tuning job {job_id} finished with status: {status}')# List all fine-tuning jobs for this resource.
print('Checking other fine-tune jobs for this resource.')
response = client.fine_tuning.jobs.list()
print(f'Found {len(response.data)} fine-tune jobs.')
输出:
{"hyperparameters": {"n_epochs": 2},"status": "running","model": "gpt-35-turbo-0613","training_file": "file-9ace76cb11f54fdd8358af27abf4a3ea","validation_file": "file-70a3f525ed774e78a77994d7a1698c4b","id": "ftjob-0f4191f0c59a4256b7a797a3d9eed219","created_at": 1695307968,"updated_at": 1695310376,"object": "fine_tuning.job"
}
Elapsed time: 40 minutes 45 seconds
Status: running
需要一个多小时才能完成训练的情况并不罕见。 训练完成后,输出消息将更改为:
Fine-tuning job ftjob-b044a9d3cf9c4228b5d393567f693b83 finished with status: succeeded
Checking other fine-tuning jobs for this resource.
Found 2 fine-tune jobs.
若要获取完整结果,请运行以下命令:
- OpenAI Python 1.x
#Retrieve fine_tuned_model nameresponse = client.fine_tuning.jobs.retrieve(job_id)print(response.model_dump_json(indent=2))
fine_tuned_model = response.fine_tuned_model
部署微调的模型
与本教程中前面的 Python SDK 命令不同,引入配额功能后,模型部署必须使用 [REST API]完成,这需要单独的授权、不同的 API 路径和不同的 API 版本。
或者,可以使用任何其他常见部署方法(例如 Azure OpenAI Studio 或 [Azure CLI])来部署微调模型。
variable | 定义 |
---|---|
token | 可通过多种方式生成授权令牌。 初始测试的最简单方法是从 Azure 门户启动 Cloud Shell。 然后运行 az account get-access-token 。 可以将此令牌用作 API 测试的临时授权令牌。 建议将其存储在新的环境变量中 |
订阅 | 关联的 Azure OpenAI 资源的订阅 ID |
resource_group | Azure OpenAI 资源的资源组名称 |
resource_name | Azure OpenAI 资源名称 |
model_deployment_name | 新微调模型部署的自定义名称。 这是在进行聊天补全调用时将在代码中引用的名称。 |
fine_tuned_model | 请从上一步的微调作业结果中检索此值。 该字符串类似于 gpt-35-turbo-0613.ft-b044a9d3cf9c4228b5d393567f693b83 。 需要将该值添加到 deploy_data json。 |
import json
import requeststoken= os.getenv("TEMP_AUTH_TOKEN")
subscription = "<YOUR_SUBSCRIPTION_ID>"
resource_group = "<YOUR_RESOURCE_GROUP_NAME>"
resource_name = "<YOUR_AZURE_OPENAI_RESOURCE_NAME>"
model_deployment_name ="YOUR_CUSTOM_MODEL_DEPLOYMENT_NAME"deploy_params = {'api-version': "2023-05-01"}
deploy_headers = {'Authorization': 'Bearer {}'.format(token), 'Content-Type': 'application/json'}deploy_data = {"sku": {"name": "standard", "capacity": 1}, "properties": {"model": {"format": "OpenAI","name": "<YOUR_FINE_TUNED_MODEL>", #retrieve this value from the previous call, it will look like gpt-35-turbo-0613.ft-b044a9d3cf9c4228b5d393567f693b83"version": "1"}}
}
deploy_data = json.dumps(deploy_data)request_url = f'https://management.azure.com/subscriptions/{subscription}/resourceGroups/{resource_group}/providers/Microsoft.CognitiveServices/accounts/{resource_name}/deployments/{model_deployment_name}'print('Creating a new deployment...')r = requests.put(request_url, params=deploy_params, headers=deploy_headers, data=deploy_data)print(r)
print(r.reason)
print(r.json())
可以在 Azure OpenAI Studio 中检查部署进度:
在处理部署微调模型时,此过程需要一些时间才能完成的情况并不罕见。
使用已部署的自定义模型
部署微调后的模型后,可以使用该模型,就像使用 Azure OpenAI Studio 的聊天平台中的任何其他已部署模型一样,或通过聊天完成 API 中来使用它。 例如,可以向已部署的模型发送聊天完成调用,如以下 Python 示例中所示。 可以继续对自定义模型使用相同的参数,例如温度和 max_tokens,就像对其他已部署的模型一样。
- OpenAI Python 1.x
import os
from openai import AzureOpenAIclient = AzureOpenAI(azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), api_key=os.getenv("AZURE_OPENAI_KEY"), api_version="2023-05-15"
)response = client.chat.completions.create(model="gpt-35-turbo-ft", # model = "Custom deployment name you chose for your fine-tuning model"messages=[{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Does Azure OpenAI support customer managed keys?"},{"role": "assistant", "content": "Yes, customer managed keys are supported by Azure OpenAI."},{"role": "user", "content": "Do other Azure AI services support this too?"}]
)print(response.choices[0].message.content)
删除部署
与其他类型的 Azure OpenAI 模型不同,微调/自定义模型在部署后会产生关联的每小时托管费用。 强烈建议你在完成本教程并针对微调后的模型测试了一些聊天完成调用后,删除模型部署。
删除部署不会对模型本身产生任何影响,因此你可以随时重新部署为本教程训练的微调模型。
可以通过 [REST API]、[Azure CLI]或其他支持的部署方法删除 Azure OpenAI Studio 中的部署。
关注TechLead,分享AI全维度知识。作者拥有10+年互联网服务架构、AI产品研发经验、团队管理经验,同济本复旦硕,复旦机器人智能实验室成员,阿里云认证的资深架构师,项目管理专业人士,上亿营收AI产品研发负责人
相关文章:

Azure Machine Learning - Azure OpenAI GPT 3.5 Turbo 微调教程
本教程将引导你在Azure平台完成对 gpt-35-turbo-0613 模型的微调。 关注TechLead,分享AI全维度知识。作者拥有10年互联网服务架构、AI产品研发经验、团队管理经验,同济本复旦硕,复旦机器人智能实验室成员,阿里云认证的资深架构师&…...

运维大模型探索之 Text2PromQL 问答机器人
作者:陈昆仪(图杨) 大家下午好,我是来自阿里云可观测团队的算法工程师陈昆仪。今天分享的主题是“和我交谈并获得您想要的PromQL”。今天我跟大家分享在将AIGC技术运用到可观测领域的探索。 今天分享主要包括5个部分:…...
虚拟机VMware:变动ip修改固定ip
1、配置ip地址 vi /etc/sysconfig/network-scripts/ifcfg-ens33修改为: 修改如下:TYPE"Ethernet" # 网络类型为以太网 BOOTPROTO"static" # 手动分配ip NAME"ens33" # 网卡…...

Docker部署Nexus Maven私服并实现远程访问Nexus界面
🎬 鸽芷咕:个人主页 🔥 个人专栏: 《linux深造日志》《粉丝福利》 ⛺️生活的理想,就是为了理想的生活! 文章目录 1. Docker安装Nexus2. 本地访问Nexus3. Linux安装Cpolar4. 配置Nexus界面公网地址5. 远程访问 Nexus界面6. 固定N…...

苏州科技大学计算机817程序设计(java) 学习笔记
之前备考苏州科技大学计算机(专业课:817程序设计(java))。 学习Java和算法相关内容,现将笔记及资料统一整理归纳移至这里。 部分内容不太完善,欢迎提议。 目录 考情分析 考卷题型 刷题攻略…...

虚幻学习笔记22—C++同步和异步加载
一、前言 之前提到的静态和动态加载都是同步的加载,同时其中的引用基本都是硬引用。如果资源比较大的话会出现卡顿的现象,下面将介绍一种异步加载的方式。同时,还将介绍一种区别与之前的Load的方法。 在说明同步和异步加载之前需要先讲一下虚…...

华清远见嵌入式学习——ARM——作业3
作业要求: 代码效果图: 代码: led.h #ifndef __LED_H__ #define __LED_H__#define RCC_GPIO (*(unsigned int *)0x50000a28) #define GPIOE_MODER (*(unsigned int *)0x50006000) #define GPIOF_MODER (*(unsigned int *)0x50007000) #defi…...
19.JavaSE
一、JavaSE。 (1)IO流。 1.字节字符流 2.标准流打印流对象流 (2)集合。 1.List/Set/Queue/Map集合 2.properties集合 (3)多线程。 1.线程创建的…...
仓库管理用什么软件
仓库管理是一个非常重要的话题,大到企业,小到个人,只要有货物的往来就会有仓库方面的管理,最为典型的就是货物的进出库存管理,这也是最为基础的仓库管理内容,那么仓库管理要用什么软件,从不同的…...
飞天使-k8s知识点8-kubernetes资源对象-编写中
文章目录 资源对象是k8s核心概念 资源对象是k8s核心概念 查看防火墙规则 32002 端口的去向 [rootkubeadm-master1 ~]# iptables -t nat -vnL |grep 32000 0 KUBE-MARK-MASQ tcp -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes-dashboard/…...
Oracle Create user
sqlplus /nolog conn sys/pw123456orcl as sysdba CREATE USER zengwenfeng IDENTIFIED BY zengwenfeng ; GRANT ALL PRIVILEGES TO zengwenfeng ; COMMIT; C:\Users\Administrator>sqlplus /nologSQL*Plus: Release 11.2.0.1.0 Production on 星期日 12月 24 21:38:24 20…...

树莓派,mediapipe,Picamera2利用舵机云台追踪人手(PID控制)
一、项目目标 追踪人手大拇指指尖: 当人手移动时,摄像头通过控制两个伺服电机(分别是偏航和俯仰)把大拇指指尖放到视界的中心位置,本文采用了PID控制伺服电机 Mediapipe Hand简介 MediaPipe 手部标志任务可检测图像…...

DQL查询数据(超重点)以及distinct(去重)
DQL(Data Query Language:数据查询语言) 1.所有查询操作都用 SELECT 2.无论是简单的查询还是复杂的查询它都能做 3.数据库中最核心的语言,最重要的语句 4.使用频率最高的语句 语法: SELECT 字段1,字段2,……FROM 表 有时候…...

【网络奇缘】——奈氏准则和香农定理从理论到实践一站式服务|计算机网络
🌈个人主页: Aileen_0v0 🔥热门专栏: 华为鸿蒙系统学习|计算机网络|数据结构与算法 💫个人格言:"没有罗马,那就自己创造罗马~" 目录 失真 - 信号的变化 影响信号失真的因素: 编辑 失真的一种现象:码间…...
MongoDB 根据 _id 获取记录的创建时间并回填记录中
MongoDB 集合 test1,有字段 _id,createTime,createTimeStr,name字段 , 查询createTime不为空的,根据 _id 生成该条记录的创建时间时间戳并填写到字段 createTime 字段中 ,并打印时间戳 // 查询 createTime…...

【开源】基于JAVA语言的独居老人物资配送系统
目录 一、摘要1.1 项目介绍1.2 项目录屏 二、功能模块三、系统展示四、核心代码4.1 查询社区4.2 新增物资4.3 查询物资4.4 查询物资配送4.5 新增物资配送 五、免责说明 一、摘要 1.1 项目介绍 基于JAVAVueSpringBootMySQL的独居老人物资配送系统,包含了社区档案、…...

网络7层架构
网络 7 层架构 什么是OSI七层模型? OSI模型用于定义并理解数据从一台计算机转移到另一台计算机,在最基本的形式中,两台计算机通过网线和连接器相互连接,在网卡的帮助下共享数据,形成一个网络,但是一台计算…...

【Arthas】Arthas线上trace匿名函数/Lambda表达式/函数式接口
前言 Arthas是一个非常牛B的东西,我非常喜欢用,特别是在定位线上问题的时候,牛逼大发! 非常建议所有Java玩家都去学习一下 阅读对象 了解并使用过Arthas了解并使用过trace命令 先说结论 先说结论,lambda表达式的追…...
阿里云“块存储”是系统盘和数据盘的意思
阿里云“块存储”是什么意思?块存储是指阿里云服务器的系统盘或数据盘。块存储EBS(Elastic Block Storage)是为云服务器ECS提供的低时延、持久性、高可靠的块级随机存储。块存储支持在可用区内自动复制您的数据,防止意外硬件故障导…...

AI赋能金融创新:ChatGPT引领量化交易新时代
文章目录 一、引言二、ChatGPT与量化交易的融合三、实践应用:ChatGPT在量化交易中的成功案例四、挑战与前景五、结论《AI时代Python量化交易实战:ChatGPT让量化交易插上翅膀》📚→ [当当](http://product.dangdang.com/29658180.html) | [京东…...
Golang dig框架与GraphQL的完美结合
将 Go 的 Dig 依赖注入框架与 GraphQL 结合使用,可以显著提升应用程序的可维护性、可测试性以及灵活性。 Dig 是一个强大的依赖注入容器,能够帮助开发者更好地管理复杂的依赖关系,而 GraphQL 则是一种用于 API 的查询语言,能够提…...
Nginx server_name 配置说明
Nginx 是一个高性能的反向代理和负载均衡服务器,其核心配置之一是 server 块中的 server_name 指令。server_name 决定了 Nginx 如何根据客户端请求的 Host 头匹配对应的虚拟主机(Virtual Host)。 1. 简介 Nginx 使用 server_name 指令来确定…...
uniapp中使用aixos 报错
问题: 在uniapp中使用aixos,运行后报如下错误: AxiosError: There is no suitable adapter to dispatch the request since : - adapter xhr is not supported by the environment - adapter http is not available in the build 解决方案&…...

多种风格导航菜单 HTML 实现(附源码)
下面我将为您展示 6 种不同风格的导航菜单实现,每种都包含完整 HTML、CSS 和 JavaScript 代码。 1. 简约水平导航栏 <!DOCTYPE html> <html lang"zh-CN"> <head><meta charset"UTF-8"><meta name"viewport&qu…...

智能分布式爬虫的数据处理流水线优化:基于深度强化学习的数据质量控制
在数字化浪潮席卷全球的今天,数据已成为企业和研究机构的核心资产。智能分布式爬虫作为高效的数据采集工具,在大规模数据获取中发挥着关键作用。然而,传统的数据处理流水线在面对复杂多变的网络环境和海量异构数据时,常出现数据质…...
【生成模型】视频生成论文调研
工作清单 上游应用方向:控制、速度、时长、高动态、多主体驱动 类型工作基础模型WAN / WAN-VACE / HunyuanVideo控制条件轨迹控制ATI~镜头控制ReCamMaster~多主体驱动Phantom~音频驱动Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation速…...

招商蛇口 | 执笔CID,启幕低密生活新境
作为中国城市生长的力量,招商蛇口以“美好生活承载者”为使命,深耕全球111座城市,以央企担当匠造时代理想人居。从深圳湾的开拓基因到西安高新CID的战略落子,招商蛇口始终与城市发展同频共振,以建筑诠释对土地与生活的…...

[ACTF2020 新生赛]Include 1(php://filter伪协议)
题目 做法 启动靶机,点进去 点进去 查看URL,有 ?fileflag.php说明存在文件包含,原理是php://filter 协议 当它与包含函数结合时,php://filter流会被当作php文件执行。 用php://filter加编码,能让PHP把文件内容…...
探索Selenium:自动化测试的神奇钥匙
目录 一、Selenium 是什么1.1 定义与概念1.2 发展历程1.3 功能概述 二、Selenium 工作原理剖析2.1 架构组成2.2 工作流程2.3 通信机制 三、Selenium 的优势3.1 跨浏览器与平台支持3.2 丰富的语言支持3.3 强大的社区支持 四、Selenium 的应用场景4.1 Web 应用自动化测试4.2 数据…...

算法打卡第18天
从中序与后序遍历序列构造二叉树 (力扣106题) 给定两个整数数组 inorder 和 postorder ,其中 inorder 是二叉树的中序遍历, postorder 是同一棵树的后序遍历,请你构造并返回这颗 二叉树 。 示例 1: 输入:inorder [9,3,15,20,7…...