【2025校招】4399 NLP算法工程师笔试题

⏰ 时间：2024/08/19
🔄 输入输出：ACM格式
⏳ 时长：2h

本试卷分为单选，自我评价题，编程题

单选和自我评价这里不再介绍，4399的编程题一如既往地抽象，明明是NLP岗位的笔试题，却考了OpenCV相关的知识。btw，跟网友讨论了下，4399似乎不同时间节点的笔试题是一样的？？？

1. 第一题

第一题是LC原题：441. 排列硬币，题目和题解请前往LC查看。

2. 第二题

题目描述

请使用OpenCV库编写程序，实现在视频文件中实时追踪一个人手持手机绿幕的四个顶点的坐标。

要求

1. 使用颜色分割技术检测绿幕区域。(8分)
2. 使用适当的方法（如轮廓检测）找到绿幕的四个顶点。(10分)
3. 在视频帧中标记出这四个顶点。(8分)

手机绿幕指：手机屏幕显示全绿色图片，用于后期处理替换为其他界面，绿色范围：lower_green = np.array([35, 100, 100])，upper_green = np.array([85, 255, 255])。

测试用例

输入：green_screen_track.mp4

输出：带顶点标记的视频序列帧图片

---

题解

import cv2
import numpy as nplower_green = np.array([35, 100, 100])
upper_green = np.array([85, 255, 255])def get_largest_contour(contours):""" 获取最大轮廓 """max_contour = max(contours, key=cv2.contourArea)return max_contourdef get_four_vertices(contour):""" 近似轮廓为四边形 """epsilon = 0.02 * cv2.arcLength(contour, True)approx = cv2.approxPolyDP(contour, epsilon, True)if len(approx) == 4:return approx.reshape(4, 2)else:return Nonedef main(video_path):cap = cv2.VideoCapture(video_path)while cap.isOpened():ret, frame = cap.read()if not ret:breakhsv_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)mask = cv2.inRange(hsv_frame, lower_green, upper_green)contours, _ = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)if contours:largest_contour = get_largest_contour(contours)vertices = get_four_vertices(largest_contour)if vertices is not None:for (x, y) in vertices:cv2.circle(frame, (x, y), 5, (0, 0, 255), -1)cv2.polylines(frame, [vertices], isClosed=True, color=(0, 255, 0), thickness=2)cv2.imshow('Green Screen Tracking', frame)if cv2.waitKey(1) & 0xFF == ord('q'):breakcap.release()cv2.destroyAllWindows()if __name__ == "__main__":video_path = 'green_screen_track.mp4'main(video_path)

3. 第三题

You can use Chinese to answer the questions.

Problem Description

You need to use the Swin Transformer model to train a binary classifier to identify whether an image contains a green screen. Green screens are commonly used in video production and photography for background replacement in post-production. Your task is to write a program that uses the Swin Transformer model to train and evaluate the performance of this classifier.

Input Data

1. Training Dataset: A set of images, including images with and without green screens.
2. Labels: Labels for each image, where 0 indicates no green screen and 1 indicates the presence of a green screen.

Output Requirements

1. Trained Model: Train a binary classifier using the Swin Transformer model.
2. Model Evaluation: Evaluate the model’s accuracy, precision, recall, and F1-score on a validation or test set.

Programming Requirements

1. Data Preprocessing: Including image loading, normalization, and label processing.
2. Model Definition: Using the Swin Transformer model.
3. Training Process: Including loss function, optimizer, and training loop.
4. Evaluation Process: Evaluate the model’s performance on the validation or test set.
5. Results Presentation: Output evaluation metrics and visualize some prediction results.

Here is a sample code framework to help you get started:

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader, Dataset
from torchvision import transforms, datasets
from swin_transformer_pytorch import SwinTransformer
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
from PIL import Image# Dataset class definition
class GreenScreenDataset(Dataset):def __init__(self, image_paths, labels, transform=None):self.image_paths = image_pathsself.labels = labelsself.transform = transformdef __len__(self):return len(self.image_paths)def __getitem__(self, idx):image = Image.open(self.image_paths[idx]).convert('RGB')label = self.labels[idx]if self.transform:image = self.transform(image)return image, label# Data preprocessing, please define transform
# TODO# Load datasets
train_dataset = GreenScreenDataset(train_image_paths, train_labels, transform=transform)
train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)val_dataset = GreenScreenDataset(val_image_paths, val_labels, transform=transform)
val_loader = DataLoader(val_dataset, batch_size=32, shuffle=False)# Define the SwinTransformer model
# TODO# Loss function and optimizer
criterion = nn.CrossEntropyLoss()
# TODO# Training process
def train(model, train_loader, criterion, optimizer, num_epochs=10):model.train()for epoch in range(num_epochs):running_loss = 0.0for images, labels in train_loader:# TODO: forward pass, compute loss, backpropagation, optimizer steprunning_loss += loss.item()print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {running_loss/len(train_loader):.4f}')# Evaluation process
def evaluate(model, val_loader):model.eval()all_preds = []all_labels = []with torch.no_grad():for images, labels in val_loader:outputs = model(images)_, preds = torch.max(outputs, 1)all_preds.extend(preds.cpu().numpy())all_labels.extend(labels.cpu().numpy())accuracy = accuracy_score(all_labels, all_preds)# TODO: Calculate precision, recall, and F1-scoreprint(f'Accuracy: {accuracy:.4f}, Precision: {precision:.4f}, Recall: {recall:.4f}, F1-score: {f1:.4f}')# Train the model
train(model, train_loader, criterion, optimizer, num_epochs=10)# Evaluate the model
evaluate(model, val_loader)

---

题解

该问题要求训练一个基于Swin Transformer模型的二分类器，用以识别图像中是否包含绿幕。解决方案涉及数据预处理、模型设计、训练和评估等多个环节。

首先，在数据预处理阶段，图像需要被调整大小并进行归一化，以满足Swin Transformer的输入需求。此外，数据集中的标签是二值化的，分别代表有无绿幕（0表示无绿幕，1表示有绿幕），确保数据集类能够准确处理这些标签是至关重要的。在模型设计上，使用了预训练的Swin Transformer模型，并针对二分类任务进行了微调。输出层被替换为一个具有两个节点的全连接层，分别对应两个类别。通过这种方式，模型能够有效地适应二分类任务。训练过程采用了标准的训练循环，设置了损失函数和优化器，并使用学习率调度器动态调整学习率。此外，为了防止过拟合，模型在训练过程中还应用了正则化技术，如dropout。在模型评估阶段，除了准确率，还使用了精确率、召回率和F1分数等指标，以全面评估模型在二分类任务中的表现。同时，为了更直观地展示模型效果，选择了一些样本图像进行可视化，显示它们的预测结果与实际标签的对比。

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader, Dataset
from torchvision import transforms
from swin_transformer_pytorch import SwinTransformer
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
from PIL import Image
import matplotlib.pyplot as plt
import numpy as np# 数据集类定义
class GreenScreenDataset(Dataset):def __init__(self, image_paths, labels, transform=None):self.image_paths = image_pathsself.labels = labelsself.transform = transformdef __len__(self):return len(self.image_paths)def __getitem__(self, idx):image = Image.open(self.image_paths[idx]).convert('RGB')label = self.labels[idx]if self.transform:image = self.transform(image)return image, torch.tensor(label, dtype=torch.long)# 数据预处理
transform = transforms.Compose([transforms.Resize((224, 224)),transforms.ToTensor(),transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])train_dataset = GreenScreenDataset(train_image_paths, train_labels, transform=transform)
train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)val_dataset = GreenScreenDataset(val_image_paths, val_labels, transform=transform)
val_loader = DataLoader(val_dataset, batch_size=32, shuffle=False)model = SwinTransformer(hidden_dim=96,layers=(2, 2, 6, 2),num_heads=(3, 6, 12, 24),num_classes=2,window_size=7,input_resolution=224
)
model = model.to(torch.device('cuda' if torch.cuda.is_available() else 'cpu'))criterion = nn.CrossEntropyLoss()
optimizer = optim.AdamW(model.parameters(), lr=1e-4, weight_decay=0.01)
scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=5, gamma=0.1)# 训练
def train(model, train_loader, criterion, optimizer, scheduler, num_epochs=10):model.train()for epoch in range(num_epochs):running_loss = 0.0for images, labels in train_loader:images, labels = images.to(device), labels.to(device)optimizer.zero_grad()outputs = model(images)loss = criterion(outputs, labels)loss.backward()optimizer.step()running_loss += loss.item()scheduler.step()print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {running_loss/len(train_loader):.4f}')# 模型评估
def evaluate(model, val_loader):model.eval()all_preds = []all_labels = []with torch.no_grad():for images, labels in val_loader:images, labels = images.to(device), labels.to(device)outputs = model(images)_, preds = torch.max(outputs, 1)all_preds.extend(preds.cpu().numpy())all_labels.extend(labels.cpu().numpy())accuracy = accuracy_score(all_labels, all_preds)precision = precision_score(all_labels, all_preds)recall = recall_score(all_labels, all_preds)f1 = f1_score(all_labels, all_preds)print(f'Accuracy: {accuracy:.4f}, Precision: {precision:.4f}, Recall: {recall:.4f}, F1-score: {f1:.4f}')return all_preds, all_labels# 可视化
def visualize_predictions(val_loader, model):model.eval()images, labels = next(iter(val_loader))images, labels = images.to(device), labels.to(device)outputs = model(images)_, preds = torch.max(outputs, 1)images = images.cpu().numpy()preds = preds.cpu().numpy()labels = labels.cpu().numpy()# 可视化前6个样本plt.figure(figsize=(12, 8))for i in range(6):plt.subplot(2, 3, i + 1)image = np.transpose(images[i], (1, 2, 0))image = image * np.array([0.229, 0.224, 0.225]) + np.array([0.485, 0.456, 0.406])  # 反归一化image = np.clip(image, 0, 1)plt.imshow(image)plt.title(f'Pred: {preds[i]}, Actual: {labels[i]}')plt.axis('off')plt.show()device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
train(model, train_loader, criterion, optimizer, scheduler, num_epochs=10)
all_preds, all_labels = evaluate(model, val_loader)
visualize_predictions(val_loader, model)