[深度学习]--分类问题的排查错误的流程

原因复现：
原生的.pt 好使， 转化后的 CoreML不好使， 分类有问题。

yolov8 格式的支持情况

                   Format     Argument           Suffix    CPU    GPU
0                 PyTorch            -              .pt   True   True
1             TorchScript  torchscript     .torchscript   True   True
2                    ONNX         onnx            .onnx   True   True
3                OpenVINO     openvino  _openvino_model   True  False
4                TensorRT       engine          .engine  False   True
5                  CoreML       coreml       .mlpackage   True  False
6   TensorFlow SavedModel  saved_model     _saved_model   True   True
7     TensorFlow GraphDef           pb              .pb   True   True
8         TensorFlow Lite       tflite          .tflite   True  False
9     TensorFlow Edge TPU      edgetpu  _edgetpu.tflite   True  False
10          TensorFlow.js         tfjs       _web_model   True  False
11           PaddlePaddle       paddle    _paddle_model   True   True
12                   NCNN         ncnn      _ncnn_model   True   True

这里可以看到CoreML 只支持cpu， 尼玛tflite也是只支持cpu的

def test_coreml():detect_weight = '/home/justin/Desktop/code/python_project/Jersey-Number/runs/detect/train64/weights/best.pt'model_detect = YOLO(detect_weight)results = model_detect(source="/home/justin/Desktop/code/python_project/Jersey-Number/zr_yz.MP4",stream=True,classes=[3])class_weight = '/home/justin/Desktop/code/python_project/Jersey-Number/runs/classify/train7/weights/best.mlpackage'class_weight = '/home/justin/Desktop/code/python_project/Jersey-Number/runs/classify/train7/weights/best.mlpackage'model_class = YOLO(class_weight)# 要使用的字体fontFace = cv2.FONT_HERSHEY_SIMPLEXfontScale = 3thickness = 1img_count = 0for result in results:img_count+=1if img_count == 6:a = 1boxes = result.boxes  # Boxes object for bounding box outputsfor box in boxes:cls = box.cls.item()conf = box.conf.item()if conf > 0.5:x1,y1,x2,y2 = box.xyxy.tolist()[0]x1,y1,x2,y2 = int(x1),int(y1),int(x2),int(y2)orig_img = result.orig_img# H,W = orig_img.orig_shapecv2.imwrite("/home/justin/Desktop/code/python_project/Jersey-Number/runs/imgs"+"{:06d}-raw.jpg".format(img_count),orig_img)cropped_image = orig_img[y1:y2,x1:x2]# res_number_class = model_class(cropped_image,save_txt=True,save=True)res_number_class = model_class(cropped_image, device = "cpu")cv2.rectangle(orig_img, (int(x1), int(y1)), (int(x2), int(y2)), (255, 0, 0), 2) for r in res_number_class:if hasattr(r,"probs"):if r.probs.top1conf.item() > 0.2:class_name = r.names[r.probs.top1](width, height), bottom = cv2.getTextSize(class_name, fontFace, fontScale=fontScale, thickness=thickness)cv2.putText(orig_img, class_name+" conf:"+str(r.probs.top1conf.item()), (x1 - width, y1-height), fontFace, fontScale, color=(0, 0, 255), thickness=thickness,lineType=cv2.LINE_AA)cv2.imwrite("/home/justin/Desktop/code/python_project/Jersey-Number/runs/imgs"+"{:06d}.jpg".format(img_count),orig_img)

报错的这句话值得看一眼：
sklearn不支持，tensorflow和torch没测试过，可能会有问题。 先跑跑再说吧

Loading /home/justin/Desktop/code/python_project/Jersey-Number/runs/classify/train7/weights/best.mlpackage for CoreML inference...
scikit-learn version 1.4.2 is not supported. Minimum required version: 0.17. Maximum required version: 1.1.2. Disabling scikit-learn conversion API.
TensorFlow version 2.13.1 has not been tested with coremltools. You may run into unexpected errors. TensorFlow 2.12.0 is the most recent version that has been tested.
Torch version 2.3.0+cu121 has not been tested with coremltools. You may run into unexpected errors. Torch 2.1.0 is the most recent version that has been tested.

所以还要降级，真是麻烦，tensorflow是因为要转android侧的模型。
这里要给个参数，来指定cpu复现
res_number_class = model_class(cropped_image, device = “cpu”)

这意思是不能用pytorch 跑了吗？ @todo， 然后用pytorch 2.0的环境试一下看看怎么样？@todo,
核心诉求是要把coreml的模型加载起来，看看是不是存在一样的错误

Exception has occurred: Exception
Model prediction is only supported on macOS version 10.13 or later.File "/home/justin/Desktop/code/python_project/Jersey-Number/zr_yz.py", line 76, in test_coremlres_number_class = model_class(cropped_image, device = "cpu")^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "/home/justin/Desktop/code/python_project/Jersey-Number/zr_yz.py", line 88, in <module>test_coreml()
Exception: Model prediction is only supported on macOS version 10.13 or later.

detect 参数
detect_conf = 0.5172230005264282
切割位置： x1,y1,x2,y2
1. 原始位置：[1648.0953369140625, 882.2176513671875, 1682.9732666015625, 980.842041015625]
2.强制转成int 为后面切出这个区域做准备(1648, 882, 1682, 980)分类输出结果：top1:64top1conf:tensor(0.9994, device='cuda:0')top5:[64, 53, 10, 0, 20]top5conf:tensor([9.9943e-01, 4.8942e-04, 1.9284e-05, 1.8095e-05, 8.8464e-06], device='cuda:0')

垃圾

shit CoreML模型只能在mac 上跑， 而且只能用CoreMl 跑么？？？ @todo???

确实只能在mac上跑
ref：
coreml的文档：
https://developer.apple.com/documentation/coreml
coremltool的文档：
https://apple.github.io/coremltools/docs-guides/
一段python代码：

import coremltools as ct
import PIL
import torch
import numpy as npdef get_x1y1x2y2(coordinate,img):width,height = img.size()center_x = int(coordinate[0] * width)center_y = int(coordinate[1] * height)img_w = int(coordinate[2]*width)img_h = int(coordinate[3]*height)return center_x, center_y, img_w, img_hdef ml_test_detect():mlmodel = ct.models.MLModel('/Users/smkj/Desktop/Code/justin/head_person_hoop_number_v8n.mlpackage')print(mlmodel)img = PIL.Image.open("/Users/smkj/Desktop/Code/justin/imgs000006-raw.jpg").resize((640,384))res = mlmodel.predict({"image":img})confidence_max_list = list(np.array(res['confidence']).argmax(axis=1))# array([0.86775684, 0.8630705 , 0.01861118, 0.09405255], dtype=float32)for row_index, class_id in enumerate(confidence_max_list):if class_id == 3:coordinate = res['coordinates'][row_index]x1,y1,x2,y2 = 555 - 12 / 2, 333  - 36 / 2, 555 + 12/2, 333 + 36/2im=img.crop((x1, y1, x2, y2))im.save("bbb.jpg")print(res)
# print(img)
def ml_test_classify():img = PIL.Image.open("/Users/smkj/Desktop/Code/justin/bbb.jpg").resize((64,64))mlmodel = ct.models.MLModel('/Users/smkj/Desktop/Code/justin/classification.mlpackage')res = mlmodel.predict({"image":img})max_key = max(res['classLabel_probs'], key=res['classLabel_probs'].get)print("class_name:",max_key, "confidence:",res['classLabel_probs'].get(max_key))a = 1
ml_test_classify()

在mac上安装opencv实在是太费劲了，各位自求多福吧！
用这个可以替代opencv： pip install pillow

置信度也是99.99

coreml不爽的点是必须要固定尺寸??? @todo 也许是我用惯了动态尺寸的原因。 anyway，今天调试了一天，在两个电脑上装了环境，算是搞定了。！！！