如何实现Mask Track RCNN

一、配置环境1. 环境选择的是pytorch 2.0.1cuda118conda env list #查看当前环境 conda create --name openmmlab python3.8 -y conda activate openmmlab pip install torch2.0.1 torchvision0.15.2 torchaudio2.0.2 --index-url https://download.pytorch.org/whl/cu118 --extra-index-url https://pypi.tuna.tsinghua.edu.cn/simple2. 其次安装 MMEngine 、MMCV。pip install -U openmim mim install mmengine mim install mmcv2.0.0 mim install mmdet3.0.03. 之后安装MMdetectiongit clone https://github.com/open-mmlab/mmdetection.git cd mmdetection pip install -v -e . # -v means verbose, or more output # -e means installing a project in editable mode, # thus any local modifications made to the code will take effect without reinstallation.二、数据准备将labelme标注出来的文件imagejson转换成YoutubeVIS 2019格式python tools/dataset_converters/labelme2video.py将YoutubeVIS 2019数据集转化成COCO 视频格式python ./tools/dataset_converters/youtubevis2coco.py -i ./data/youtube_vis_2019 -o ./data/youtube_vis_2019/annotations --version 2019三、模型训练实验中跑了ResNet50、ResNet101、RexNet101三种backbone下的模型。batchsize2epoch50.#训练代码 4块GPU #ResNet50 bash tools/dist_train.sh configs/masktrack_rcnn/ME_masktrack_inial.py 4 #ResNet101 bash tools/dist_train.sh configs/masktrack_rcnn/ME_masktrack_101.py 4 #RexNet101 bash tools/dist_train.sh configs/masktrack_rcnn/ME_masktrack_x101.py 4 #如果要使用随机数种子的话命令变成 bash tools/dist_train.sh configs/masktrack_rcnn/ME_masktrack_inial.py 4 \ --seed 0 \ --work-dir work_dirs/ME_masktrack_round2_seed0四、推理一显示图像#使用mot_demo.py输入的数据有视频、config配置、模型权重、保存成哪个mp4、fps设置。 python /hy-tmp/demo/mot_demo.py data/youtube_vis_2019me/test/JPEGImages/new_downcell3 configs/masktrack_rcnn/masktrack-rcnn_mask-rcnn_r50_fpn_8xb1-12e_youtubevis2019.py --checkpoint /hy-tmp/work_dirs/masktrack-rcnn_r50/epoch_50.pth --show --out visbool_cell2.mp4 --fps 5二计算mAP和mAR重点#将评估数据先打印出一个json文件result.json以压缩包的形式放在里面 python tools/test_tracking.py configs/masktrack_rcnn/ME_masktrack_inial.py --checkpoint work_dirs/masktrack-rcnn_r50_round1/epoch_50.pth --work-dir youtube_vis_R50 #其次将result.json下载到本地使用Google Colab运行代码输出mAP和mARresult.json下载到本地使用Google Colab计算mAP和mAR计算的代码如下这段代码在seqformer_refine中import matplotlib.pyplot as plt from pycocotools.ytvos import YTVOS from pycocotools.ytvoseval import YTVOSeval import pylab import numpy as np # 强行给 numpy 加上 float 属性欺骗旧代码 np.float float np.int int # 顺手把 int 也补上以防万一 # --------------------- vita ------------------ pylab.rcParams[figure.figsize] (10.0, 8.0) annType bbox # 放valid.json annFile /content/test.json visGt YTVOS(annFile) #initialize vis detections api resFile /content/results_maskrcnn_final.json visDt visGt.loadRes(resFile) vidIds sorted(visGt.getVidIds()) # vidIds vidIds[0:53] # vidIds vidIds[np.random.randint(53)] # running evaluation visEval YTVOSeval(visGt,visDt,annType) visEval.params.vidIds vidIds visEval.evaluate() visEval.accumulate() visEval.summarize()四、Baseline与SOTABaseline选择使用去掉Tracking Head的部分训练baseline的命令行为#训练代码 bash tools/dist_train.sh /hy-tmp/configs/mask_rcnn/my_baseline_r50.py 4 --seed 1717 --work-dir /hy-tmp/work_dirs_baseline/R50 #推理代码将图像的结果拼接成视频的结果然后再放到Google Colab里面去计算 python tools/inference_to_ytvis.py