计算机视觉之dlib人脸关键点绘制及微笑测试

dlib人脸关键点绘制及微笑测试

1 dlib人脸关键点

---

1.1 dlib

dlib 是一个强大的机器学习库，广泛用于人脸检测和人脸关键点检测。它提供了一个预训练的 68 点人脸关键点检测模型，可以准确地定位人脸的各个部位（如眼睛、鼻子、嘴巴等）

1.2 人脸关键点检测

dlib 的 68 点人脸关键点检测模型基于 HOG（Histogram of Oriented Gradients）特征和线性分类器，结合了形状预测算法。它可以检测人脸的以下区域：
下巴（0-16）
右眉毛（17-21）
左眉毛（22-26）
鼻子（27-35）
右眼（36-41）
左眼（42-47）
嘴巴（48-67）

1.3 检测模型

dlib 提供了一个预训练的 68 点人脸关键点检测模型，可以从以下链接下载：
https://github.com/davisking/dlib-models/blob/master/shape_predictor_68_face_landmarks.dat.bz2/

1.4 凸包

凸包（Convex Hull） 是计算几何中的一个重要概念，指的是在二维或更高维空间中，包含一组点的最小凸多边形或凸多面体。凸包在图像处理、计算机视觉、模式识别等领域有广泛应用，例如在人脸关键点检测中，可以用凸包来定义人脸区域的边界。

1.5 笑容检测

定义了两个函数，MAR：衡量嘴巴的张开程度，
和MJR：衡量嘴巴宽度与下巴宽度的比例，
人脸关键点如上，当微笑时嘴巴长款和脸颊长度都会发生改变，通过两个函数进行比较检测，进行判断是否微笑

def MAR(shape):x = shape[50]y = shape[50].reshape(1,2)A = euclidean_distances(shape[50].reshape(1,2),shape[58].reshape(1,2))B = euclidean_distances(shape[51].reshape(1,2),shape[57].reshape(1,2))C = euclidean_distances(shape[52].reshape(1,2),shape[56].reshape(1,2))D = euclidean_distances(shape[48].reshape(1,2),shape[54].reshape(1,2))return ((A+B+C)/3)/Ddef MJR(shape):M = euclidean_distances(shape[48].reshape(1,2),shape[54].reshape(1,2))J = euclidean_distances(shape[3].reshape(1,2),shape[13].reshape(1,2))return M/J

1.6 函数

• detector = dlib.get_frontal_face_detector() ，加载人脸检测器
• predictor = dlib.shape_predictor(‘shape_predictor_68_face_landmarks.dat’) 关键点预测器
• detector(gray, 1) ，检测人脸
  • gray检测的灰度图
  • 1 表示对图像进行上采样次数

2 人脸检测代码

---

2.1 关键点绘制

代码展示：

import cv2
import numpy as np
import dlibimg = cv2.imread('lyf.png')
detector = dlib.get_frontal_face_detector()
faces = detector(img,0)
predictor = dlib.shape_predictor('shape_predictor_68_face_landmarks.dat')
for face in faces:shape = predictor(img,face)landmarks = np.array([[p.x,p.y] for p in shape.parts()])for idx,point in enumerate(landmarks):pos = [point[0],point[1]]cv2.circle(img,pos,2,color=(0,255,0),thickness=-1)cv2.putText(img,str(idx),pos,cv2.FONT_HERSHEY_SCRIPT_SIMPLEX,0.4,(255,255,255),1,cv2.LINE_AA)
cv2.imshow('img',img)
cv2.waitKey(0)
cv2.destroyAllWindows()

运行结果：

2.2 关键点连线

代码展示：

import cv2
import numpy as np
import dlibdef drawLine(start,end):pts = shape[start:end]for l in  range(1,len(pts)):pta = tuple(pts[l-1])ptb = tuple(pts[l])cv2.line(img,pta,ptb,(0,255,0),1)def drawConvexHull(start,end):facial = shape[start:end+1]mouthHull = cv2.convexHull(facial)cv2.drawContours(img,[mouthHull],-1,(0,255,0),1)img = cv2.imread('lyf.png')
detector = dlib.get_frontal_face_detector()
faces = detector(img,0)
predictor = dlib.shape_predictor('shape_predictor_68_face_landmarks.dat')
for face in faces:shape = predictor(img,face)shape = np.array([[p.x,p.y] for p in shape.parts()])drawConvexHull(36,41)drawConvexHull(42,47)drawConvexHull(48, 59)drawConvexHull(60, 67)drawLine(0,17)drawLine(17, 22)drawLine(22, 27)drawLine(27, 36)cv2.imshow('img',img)
cv2.waitKey(0)
cv2.destroyAllWindows()

运行结果：

2.3 微笑检测

代码展示：

import cv2
import numpy as np
import dlib
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor('shape_predictor_68_face_landmarks.dat')
v = cv2.VideoCapture('jjy_dyx.mp4')
from sklearn.metrics.pairwise import euclidean_distances
from PIL import Image, ImageDraw, ImageFontdef cv2AddChineseText(img, text, position, textColor=(255, 255, 255), textSize=30):""" 向图片中添加中文 """if (isinstance(img, np.ndarray)):  # 判断是否OpenCV图片类型img = Image.fromarray(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))#实现array到image的转换draw = ImageDraw.Draw(img)# 在img图片上创建一个绘图的对象# 字体的格式fontStyle = ImageFont.truetype("simsun.ttc", textSize, encoding="utf-8")draw.text(position, text, textColor, font=fontStyle) # 绘制文本return cv2.cvtColor(np.asarray(img), cv2.COLOR_RGB2BGR)# 转换回OpenCV格式def MAR(shape):x = shape[50]y = shape[50].reshape(1,2)A = euclidean_distances(shape[50].reshape(1,2),shape[58].reshape(1,2))B = euclidean_distances(shape[51].reshape(1,2),shape[57].reshape(1,2))C = euclidean_distances(shape[52].reshape(1,2),shape[56].reshape(1,2))D = euclidean_distances(shape[48].reshape(1,2),shape[54].reshape(1,2))return ((A+B+C)/3)/Ddef MJR(shape):M = euclidean_distances(shape[48].reshape(1,2),shape[54].reshape(1,2))J = euclidean_distances(shape[3].reshape(1,2),shape[13].reshape(1,2))return M/Jwhile True:r,img = v.read()if not r:breakfaces = detector(img,0)for face in faces:shape = predictor(img,face)shape= np.array([[p.x,p.y] for p in shape.parts()])mar = MAR(shape)mjr =MJR(shape)result = '正常'print('mar:',mar,'mjr:',mjr)if mar>0.5:result = '大笑'elif mjr>0.4:result = '微笑'mouthHull = cv2.convexHull(shape[48:61])img = cv2AddChineseText(img,result,mouthHull[0,0],1)cv2.drawContours(img,[mouthHull],-1,(0,255,0),1)cv2.imshow('img', img)key = cv2.waitKey(1)if key == 32:break
v.release()
cv2.waitKey(0)
cv2.destroyAllWindows()

运行结果：