抬手、放手识别算法

在一款智能手表中， 平时手表处于息屏的状态， 用于节省功耗，延长使用时间。 在用户进行抬手的时候，其实是希望能够及时看一下时间、消息通知等信息的。这时手表应该能够检测到用户的抬手动作，自动进行屏幕的点亮。当用户垂手返回的时候，也应该识别到，及时进行屏幕的关闭。

对于这种动作的侦测，采用加速度传感器，是常见的选择。加速度传感采用固定的频率，进行X轴、Y轴、Z轴三轴数据的采集，放置在FIFO队列中， MCU可以定期去获取这些数据， 对于这些数据进行处理和特征识别。 当识别出抬手、放手时，进行相应的逻辑处理。

1. 数据

这个特征是怎么样的呢？我们可以先看一下，在反复进行抬手、放手动作时，所抓取到的数据。 

wrist_algo_timeout_handler(82): 1730099274701: wrist: i = 0, x = -8.842969, y = 2.871094, z = 0.976172
wrist_algo_timeout_handler(82): 1730099274703: wrist: i = 1, x = -8.842969, y = 2.890234, z = 0.899609
wrist_algo_timeout_handler(82): 1730099274705: wrist: i = 2, x = -8.823829, y = 2.909375, z = 0.861328
wrist_algo_timeout_handler(82): 1730099274707: wrist: i = 3, x = -8.862109, y = 2.928516, z = 0.861328
wrist_algo_timeout_handler(82): 1730099274710: wrist: i = 4, x = -8.900391, y = 2.947656, z = 0.861328
wrist_algo_timeout_handler(82): 1730099274712: wrist: i = 5, x = -8.900391, y = 2.966797, z = 0.861328
wrist_algo_timeout_handler(82): 1730099274714: wrist: i = 6, x = -8.900391, y = 2.966797, z = 0.861328
wrist_algo_timeout_handler(82): 1730099274716: wrist: i = 7, x = -8.881250, y = 2.909375, z = 0.842188
wrist_algo_timeout_handler(82): 1730099274719: wrist: i = 8, x = -8.919532, y = 2.947656, z = 0.842188
wrist_algo_timeout_handler(82): 1730099274721: wrist: i = 9, x = -8.919532, y = 2.966797, z = 0.861328
wrist_algo_timeout_handler(82): 1730099274723: wrist: i = 10, x = -8.957812, y = 2.966797, z = 0.899609
wrist_algo_timeout_handler(82): 1730099274726: wrist: i = 11, x = -8.900391, y = 2.928516, z = 0.899609
wrist_algo_timeout_handler(82): 1730099274728: wrist: i = 12, x = -8.938672, y = 2.947656, z = 0.880469
wrist_algo_timeout_handler(82): 1730099274730: wrist: i = 13, x = -8.957812, y = 2.909375, z = 0.861328
wrist_algo_timeout_handler(82): 1730099274733: wrist: i = 14, x = -8.957812, y = 2.928516, z = 0.861328
wrist_algo_timeout_handler(82): 1730099274735: wrist: i = 15, x = -8.976954, y = 2.966797, z = 0.880469
wrist_algo_timeout_handler(82): 1730099274737: wrist: i = 16, x = -8.996094, y = 3.005078, z = 0.823047
wrist_algo_timeout_handler(82): 1730099274739: wrist: i = 17, x = -9.034375, y = 3.024219, z = 0.861328
wrist_algo_timeout_handler(82): 1730099274742: wrist: i = 18, x = -8.957812, y = 3.062500, z = 0.861328
wrist_algo_timeout_handler(82): 1730099274744: wrist: i = 19, x = -8.996094, y = 3.062500, z = 0.880469
wrist_algo_timeout_handler(82): 1730099274746: wrist: i = 20, x = -8.919532, y = 3.024219, z = 0.880469
wrist_algo_timeout_handler(82): 1730099274749: wrist: i = 21, x = -8.900391, y = 3.043360, z = 0.880469
wrist_algo_timeout_handler(82): 1730099274751: wrist: i = 22, x = -8.919532, y = 3.024219, z = 0.880469
wrist_algo_timeout_handler(82): 1730099274753: wrist: i = 23, x = -8.996094, y = 3.043360, z = 0.899609
wrist_algo_timeout_handler(82): 1730099274756: wrist: i = 24, x = -8.938672, y = 3.043360, z = 0.995313
wrist_algo_timeout_handler(82): 1730099275212: wrist: i = 0, x = -8.919532, y = 3.005078, z = 1.014453
wrist_algo_timeout_handler(82): 1730099275214: wrist: i = 1, x = -8.976954, y = 3.024219, z = 1.033594
wrist_algo_timeout_handler(82): 1730099275216: wrist: i = 2, x = -8.919532, y = 3.005078, z = 1.033594
wrist_algo_timeout_handler(82): 1730099275218: wrist: i = 3, x = -8.900391, y = 3.024219, z = 1.052734
wrist_algo_timeout_handler(82): 1730099275221: wrist: i = 4, x = -8.881250, y = 2.947656, z = 1.014453
wrist_algo_timeout_handler(82): 1730099275223: wrist: i = 5, x = -8.862109, y = 2.947656, z = 1.014453
wrist_algo_timeout_handler(82): 1730099275225: wrist: i = 6, x = -8.938672, y = 2.966797, z = 0.995313
wrist_algo_timeout_handler(82): 1730099275228: wrist: i = 7, x = -8.976954, y = 3.005078, z = 0.976172
wrist_algo_timeout_handler(82): 1730099275230: wrist: i = 8, x = -8.957812, y = 3.005078, z = 0.995313
wrist_algo_timeout_handler(82): 1730099275232: wrist: i = 9, x = -8.957812, y = 2.966797, z = 0.957031

......................................................

2. 波形

对于三轴的数据， 我们可以进行波形绘制， 观察出具体的特征。

import os
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.ticker import MultipleLocator#从Log中解析出来X、Y、Z三轴的原始数据
raw_data_file = "data/data.txt"
count = 0
lineNo = 0
listX = []
listY = []
listZ = []
listV = []with open(raw_data_file, 'r', encoding='utf-8') as fread:for text in fread.readlines():lineNo += 1xyz = text.split(',')xyz.pop(0)x = xyz.pop(0)y = xyz.pop(0)z = xyz.pop(0)dataX = float(x[4:])dataY = float(y[4:])dataZ = float(z[4:])dataV = np.sqrt(dataX*dataX + dataY*dataY +dataZ*dataZ)listX.append(dataX)listY.append(dataY)listZ.append(dataZ)listV.append(dataV)count += 1print("count no : " + str(count))plt.figure()
ax=plt.subplot(1,1,1)
ax.plot(listX[0:count], label='X', color='red')
ax.plot(listY[0:count], label='Y', color='blue')
ax.plot(listZ[0:count], label='Z', color='green')ax.grid(True)
ax.xaxis.set_major_locator(MultipleLocator(10))
ax.yaxis.set_major_locator(MultipleLocator(1))
plt.show()

具体的波形，可以如下图：

.

可以截取一段，看得更加清楚：

3. 算法

主要的算法依据是，当手臂下垂时，假设X轴垂直于表盘，Z轴垂直于地面，如果处于静止状态，则X轴的值为0g， Z轴为1g，当抬手转动到用户面前的时候，这时，X轴的值为1g，Z轴的值为0g， 当垂手灭屏的时候，与之相反，所以，需要重点关注两个轴向的数据变化，则可以实现抬手、垂手这两个动作的识别。

下面我们介绍采用代码进行动作识别的过程。

# 1. 进行滤波

# 2. 进行动态阈值

# 3. 进行动态精度

# 4. 判断峰值，侦测动作类型

一些全局的变量定义如下：

class peak_value:def __init__(self):self.newMax = 0.0self.newMin = 0.0self.oldMax = 0.0self.oldMin = 0.0SAMPLE_SIZE = 100
DYNAMIC_PRECISION = 0.01
FILTER_CNT = 4
NULL = 0
UP = 1
DOWN = 2
MAXTIMEOUT = 10 #翻腕时间阈值设定为400ms
THRESHOLD = 2.0 #翻腕的值应该超过1.0g
DELAYCOUNT = 10 #应该延迟几个点进行判断
filtered_sample_x = 0.0
filter_fifo_x = []
filtered_sample_z = 0.0
filter_fifo_z = []
peak_x = peak_value()
peak_z = peak_value()
threshold_x = 0.0
threshold_z = 0.0
th_x = []
th_z = []
sample_size_x = 0
sample_size_z = 0
newSample_x = 0.0
oldSample_x = 0.0
newSample_z = 0.0
oldSample_z = 0.0
hand_on = 0
hand_off = 0
timeCount = 0
lastTimeCount_x = 0
lastTimeCount_z = 0
timeInterval_x = 0
timeInterval_z = 0
x_status = NULL
z_status = NULLdef MAX(a, b):if (a>b):return aelse:return bdef MIN(a, b):if (a<b):return aelse:return b

3.1 滤波

滤波的方式有很多种， 在这里我们采用均值滤波， 可以消除一下毛刺。不能过滤的过于平滑， 会失去特征。

#进行均值滤波
def filter_calculate_x(sample_x):global filtered_sample_x, filter_fifo_xfilter_fifo_x.pop(0)filter_fifo_x.append(sample_x)x_sum = 0.0for i in range(0, FILTER_CNT):x_sum += filter_fifo_x[i]filtered_sample_x = x_sum/FILTER_CNTdef filter_calculate_z(sample_z):global filtered_sample_z, filter_fifo_zfilter_fifo_z.pop(0)filter_fifo_z.append(sample_z)z_sum = 0.0for i in range(0, FILTER_CNT):z_sum += filter_fifo_z[i]filtered_sample_z = z_sum/FILTER_CNT

3.2 动态阈值

一个轴向上从0G到1G，另一个轴向上从1G到0G，这是比较理想的情况， 其实用户在佩戴过程中， 是不会有这么准确的值的。应该是采用动态阈值的情况，不能采用绝对的值。

#采用设置动态阈值的方式
sample_fifo_x = []
def peak_update_x(cur_sample_x):global sample_size_x, peak_x, threshold_x, sample_fifo_xsample_fifo_x.append(cur_sample_x)sample_size_x += 1if (sample_size_x >= SAMPLE_SIZE):peak_x.newMax = sample_fifo_x[0]peak_x.newMin = sample_fifo_x[0]for i in range(0, SAMPLE_SIZE):peak_x.newMax = MAX(peak_x.newMax, sample_fifo_x[i])peak_x.newMin = MIN(peak_x.newMin, sample_fifo_x[i])peak_x.oldMax = peak_x.newMaxpeak_x.oldMin = peak_x.newMinthreshold_x = (peak_x.oldMax + peak_x.oldMin)/2sample_fifo_x.pop(0)sample_fifo_z = []
def peak_update_z(cur_sample_z):global sample_size_z, peak_z, threshold_z, sample_fifo_zsample_fifo_z.append(cur_sample_z)sample_size_z += 1if (sample_size_z >= SAMPLE_SIZE):peak_z.newMax = sample_fifo_z[0]peak_z.newMin = sample_fifo_z[0]for i in range(0, SAMPLE_SIZE):peak_z.newMax = MAX(peak_z.newMax, sample_fifo_z[i])peak_z.newMin = MIN(peak_z.newMin, sample_fifo_z[i])peak_z.oldMax = peak_z.newMaxpeak_z.oldMin = peak_z.newMinthreshold_z = (peak_z.oldMax + peak_z.oldMin)/2sample_fifo_z.pop(0)

3.3 动态精度

对于相邻的点，如果变化很小， 可以丢弃掉。

#动态精度
def slide_update_x(cur_sample_x):res = Falseglobal newSample_x, oldSample_xif (abs(cur_sample_x - newSample_x) > DYNAMIC_PRECISION):oldSample_x = newSample_xnewSample_x = cur_sample_xres = Trueelse:oldSample_x = newSample_xreturn resdef slide_update_z(cur_sample_z):res = Falseglobal newSample_z, oldSample_zif (abs(cur_sample_z - newSample_z) > DYNAMIC_PRECISION):oldSample_z = newSample_znewSample_z = cur_sample_zres = Trueelse:oldSample_z = newSample_zreturn res

3.4 动作识别

对于一段数据，设置完阈值后，进行两轴向数据的判断，从而识别出抬手、放手的动作。

#姿态判断
def detect_tilt():global hand_on, hand_off, oldSample_x, oldSample_z, newSample_x, newSample_z, threshold_x, threshold_z, lastTimeCount_x, lastTimeCount_z, timeInterval_x, timeInterval_z, x_status, z_status, timeCounttimeInterval_x = timeCount - lastTimeCount_xtimeInterval_z = timeCount - lastTimeCount_zif (timeInterval_x > MAXTIMEOUT):x_status = NULLif (timeInterval_z > MAXTIMEOUT):z_status = NULLif (np.abs(newSample_x - peak_x.oldMax) >= THRESHOLD or np.abs(newSample_x - peak_x.oldMin) >= THRESHOLD):if (oldSample_x > threshold_x and newSample_x < threshold_x):x_status = DOWNlastTimeCount_x = timeCount#print("x down : " + str(timeCount))if (oldSample_x < threshold_x and newSample_x > threshold_x):x_status = UPlastTimeCount_x = timeCount#print("x up : " + str(timeCount))if (np.abs(newSample_z - peak_z.oldMax)>= THRESHOLD or np.abs(newSample_z - peak_z.oldMin) >= THRESHOLD):if (oldSample_z > threshold_z and newSample_z < threshold_z):z_status = DOWNlastTimeCount_z = timeCount#print("z down : " + str(timeCount))if (oldSample_z < threshold_z and newSample_z > threshold_z):z_status = UPlastTimeCount_z = timeCount#print("z up : " + str(timeCount))if (x_status == UP and z_status == DOWN):hand_on += 1print("timeCount : " + str(timeCount))print("hand on : " + str(hand_on))x_status = NULLz_status = NULLif (x_status == DOWN and z_status == UP):hand_off += 1print("timeCount : " + str(timeCount) )print("hand off : " + str(hand_off))x_status = NULLz_status = NULL

这几个函数的具体调用过程，如下：

#进行第一次X轴、Z轴阈值设定
peak_x.newMax = listX[0]
peak_x.newMin = listX[0]
peak_x.oldMax = listX[0]
peak_x.oldMin = listX[0]peak_z.newMax = listZ[0]
peak_z.newMin = listZ[0]
peak_z.oldMax = listZ[0]
peak_z.oldMin = listZ[0]for i in range(0, SAMPLE_SIZE):cur_sample_x = listX[i]cur_sample_z = listZ[i]peak_update_x(cur_sample_x)peak_update_z(cur_sample_z)#设定第一个滤波器的数据
for i in range(0, FILTER_CNT):cur_sample_x = listX[i]cur_sample_z = listZ[i]filter_fifo_x.append(cur_sample_x)filter_fifo_z.append(cur_sample_z)#循环获取Sample的点， 进行动态阈值的更新， 抬手+垂手动作姿势的侦测
mean_listX = []
mean_listY = []
mean_listZ = []listX_buffer = []
listZ_buffer = []for i in range(0, count):timeCount += 1cur_sample_x = listX[i]cur_sample_z = listZ[i]filter_calculate_x(cur_sample_x)filter_calculate_z(cur_sample_z)peak_update_x(filtered_sample_x)peak_update_z(filtered_sample_z)th_x.append(threshold_x)th_z.append(threshold_z)listX_buffer.append(filtered_sample_x)listZ_buffer.append(filtered_sample_z)if (timeCount > DELAYCOUNT):x = listX_buffer.pop(0)z = listZ_buffer.pop(0)mean_listX.append(x)mean_listZ.append(z)slide_update_x(x)slide_update_z(z)detect_tilt()print("hand on : " + str(hand_on))
print("hand off : " + str(hand_off))

3.5 波形示意

可以把增加了动态阈值，和滤波后的波形，与原始波形进行对比， 如下所示：

#绘制波形
#前面几个值要补齐
for i in range(0, DELAYCOUNT):mean_listX.insert(0, 0.0)mean_listZ.insert(0, 0.0)count = len(mean_listX)
plt.figure()
ax=plt.subplot(1,1,1)
ax.plot(mean_listX[0:count], label='X', color='red')
ax.plot(th_x[0:count], ls = "--", label='th_x', color = 'red')
ax.plot(mean_listZ[0:count], label='Z', color='green')
ax.plot(th_z[0:count], ls = "--", label='th_z', color = 'green')
ax.grid(True)
ax.xaxis.set_major_locator(MultipleLocator(10))
ax.yaxis.set_major_locator(MultipleLocator(1))
plt.show()