0. 引言
利用 Python 开发，对于输入的摄像头视频流，借助 Dlib 提供的检测识别模型来进行人脸识别；
首先，从摄像头中录入（裁剪）人脸图片存储到本地，然后提取特征，构建预设人脸特征；
对于 person_x，根据录入的多张人脸图片提取 128D 特征值，然后计算该人的 128D 特征均值；
然后和摄像头中实时获取到的人脸提取出的特征值，计算欧氏距离，判定是否为同一张人脸；
识别模型：基于 Dlib 的 ResNet 预训练模型（dlib_face_recognition_resnet_model_v1.dat）
识别算法：ResNet 神经网络（This model is a ResNet network with 29 conv layers. It's essentially a version of the ResNet-34 network from the paper Deep Residual Learning for Image Recognition by He, Zhang, Ren, and Sun with a few layers removed and the number of filters per layer reduced by half）
调用的模型以及大概的耗时：
博客中代码以 GitHub 为准，博客中可能没有及时更新；
# Blog :
# GitHub :
支持的功能：
人脸识别 / Face Recognition 的说明：
Wikipedia 上关于人脸识别系统 / Face Recognition System 的描述：they work by comparing selected facial features from given image with faces within a database.
本项目中就是比较 预设的人脸的特征 和 摄像头实时获取到的人脸的特征 ；
核心就是 提取 128D 人脸特征，然后计算 摄像头人脸特征 和 预设的特征脸的欧式距离，进行比对；
效果如下：
图 1 多人脸实时识别效果
1. 总体流程
先说下 人脸检测 ( Face detection ) 和 人脸识别 ( Face Recognition ) ，前者是达到检测出场景中人脸的目的就可以了，而后者不仅需要检测出人脸，还要和已有人脸数据进行比对，识别出是否在数据库中，或者进行身份标注之类处理，人脸检测和人脸识别两者有时候可能会被理解混淆；
利用“dlib_face_recognition_resnet_model_v1.dat” 这个训练好的 resnet-34 模型，提取 人脸图像的 128D 特征，然后比对不同人脸图片的 128D 特征的欧式距离，设定一个 阈值 来判断是否为同一张脸；
1 # face recognition model, the object maps human faces into 128D vectors2 facerec = dlib.face_recognition_model_v1("dlib_face_recognition_resnet_model_v1.dat")34 shape = predictor(img, dets[0])5 face_descriptor = facerec.compute_face_descriptor(img, shape)
图 2 总体设计流程
2. 源码介绍
有三块代码，分别是人脸录入，特征提取和人脸识别：
2.1 get_faces_from_camera.py / 人脸注册录入
人脸识别需要将 提取到的图像数据 和 已有图像数据 进行比对分析，所以这部分代码实现的功能就是 人脸录入；
程序会生成一个窗口，显示调用的摄像头实时获取的图像；
（关于摄像头的调用方式可以参考这里： ）；
2.1.1 GUI with OpenCV
利用 OpenCV 生成的窗口界面，用户通过键盘输入进行人脸捕获：
摄像头的调用是利用 opencv 库的 cv2.VideoCapture(0), 此处参数为 0 代表调用的是笔记本的默认摄像头，你也可以让它调用传入已有视频文件；
可以参考 如何通过 OpenCV 调用摄像头；
图 3 get_face_from_camera.py 的界面
“N”+“S”之后捕获到的一组人脸示例；
图 4 捕获到的一组人脸
源码:
1 # Copyright (C) 2020 coneypo 2 # SPDX-License-Identifier: MIT 3 4 # Author: coneypo 5 # Blog: 6 # GitHub: 7 # Mail: coneypo@foxmail.com 8 9 # 进行人脸录入 / Face register 10 11 import dlib 12 import numpy as np 13 import cv2 14 import os 15 import shutil 16 import time 17 18 # Dlib 正向人脸检测器 / Use frontal face detector of Dlib 19 detector = dlib.get_frontal_face_detector() 20 21 22 class Face_Register: 23 def __init__(self): 24 self.path_photos_from_camera = "data/data_faces_from_camera/" 25 self.font = cv2.FONT_ITALIC 26 27 self.existing_faces_cnt = 0 # 已录入的人脸计数器 / cnt for counting saved faces 28 self.ss_cnt = 0 # 录入 personX 人脸时图片计数器 / cnt for screen shots 29 self.current_frame_faces_cnt = 0 # 录入人脸计数器 / cnt for counting faces in current frame 30 31 self.save_flag = 1 # 之后用来控制是否保存图像的 flag / The flag to control if save 32 self.press_n_flag = 0 # 之后用来检查是否先按 'n' 再按 's' / The flag to check if press 'n' before 's' 33 34 # FPS 35 self.frame_time = 0 36 self.frame_start_time = 0 37 self.fps = 0 38 39 # 新建保存人脸图像文件和数据CSV文件夹 / Make dir for saving photos and csv 40 def pre_work_mkdir(self): 41 # 新建文件夹 / Create folders to save faces images and csv 42 if os.path.isdir(self.path_photos_from_camera): 43 pass 44 else: 45 os.mkdir(self.path_photos_from_camera) 46 47 # 删除之前存的人脸数据文件夹 / Delete the old data of faces 48 def pre_work_del_old_face_folders(self): 49 # 删除之前存的人脸数据文件夹, 删除 "/data_faces_from_camera/person_x/"... 50 folders_rd = os.listdir(self.path_photos_from_camera) 51 for i in range(len(folders_rd)): 52 shutil.rmtree(self.path_photos_from_camera+folders_rd[i]) 53 if os.path.isfile("data/features_all.csv"): 54 os.remove("data/features_all.csv") 55 56 # 如果有之前录入的人脸, 在之前 person_x 的序号按照 person_x+1 开始录入 / Start from person_x+1 57 def check_existing_faces_cnt(self): 58 if os.listdir("data/data_faces_from_camera/"): 59 # 获取已录入的最后一个人脸序号 / Get the order of latest person 60 person_list = os.listdir("data/data_faces_from_camera/") 61 person_num_list = [] 62 for person in person_list: 63 person_num_list.append(int(person.split('_')[-1])) 64 self.existing_faces_cnt = max(person_num_list) 65 66 # 如果第一次存储或者没有之前录入的人脸, 按照 person_1 开始录入 / Start from person_1 67 else: 68 self.existing_faces_cnt = 0 69 70 # 获取处理之后 stream 的帧数 / Update FPS of video stream 71 def update_fps(self): 72 now = time.time() 73 self.frame_time = now - self.frame_start_time 74 self.fps = 1.0 / self.frame_time 75 self.frame_start_time = now 76 77 # 生成的 cv2 window 上面添加说明文字 / PutText on cv2 window 78 def draw_note(self, img_rd): 79 # 添加说明 / Add some notes 80 cv2.putText(img_rd, "Face Register", (20, 40), self.font, 1, (255, 255, 255), 1, cv2.LINE_AA) 81 cv2.putText(img_rd, "FPS: " + str(self.fps.__round__(2)), (20, 100), self.font, 0.8, (0, 255, 0), 1, 82 cv2.LINE_AA) 83 cv2.putText(img_rd, "Faces: " + str(self.current_frame_faces_cnt), (20, 140), self.font, 0.8, (0, 255, 0), 1, cv2.LINE_AA) 84 cv2.putText(img_rd, "N: Create face folder", (20, 350), self.font, 0.8, (255, 255, 255), 1, cv2.LINE_AA) 85 cv2.putText(img_rd, "S: Save current face", (20, 400), self.font, 0.8, (255, 255, 255), 1, cv2.LINE_AA) 86 cv2.putText(img_rd, "Q: Quit", (20, 450), self.font, 0.8, (255, 255, 255), 1, cv2.LINE_AA) 87 88 # 获取人脸 / Main process of face detection and saving 89 def process(self, stream): 90 # 1. 新建储存人脸图像文件目录 / Create folders to save photos 91 self.pre_work_mkdir() 92 93 # 2. 删除 "/data/data_faces_from_camera" 中已有人脸图像文件 / Uncomment if want to delete the saved faces and start from person_1 94 if os.path.isdir(self.path_photos_from_camera): 95 self.pre_work_del_old_face_folders() 96 97 # 3. 检查 "/data/data_faces_from_camera" 中已有人脸文件 98 self.check_existing_faces_cnt() 99100 while stream.isOpened():101 flag, img_rd = stream.read() # Get camera video stream102 kk = cv2.waitKey(1)103 faces = detector(img_rd, 0) # Use Dlib face detector104105 # 4. 按下 'n' 新建存储人脸的文件夹 / Press 'n' to create the folders for saving faces106 if kk == ord('n'):107 self.existing_faces_cnt += 1108 current_face_dir = self.path_photos_from_camera + "person_" + str(self.existing_faces_cnt)109 os.makedirs(current_face_dir)110 print('\n')111 print("新建的人脸文件夹 / Create folders: ", current_face_dir)112113 self.ss_cnt = 0 # 将人脸计数器清零 / Clear the cnt of screen shots114 self.press_n_flag = 1 # 已经按下 'n' / Pressed 'n' already115116 # 5. 检测到人脸 / Face detected117 if len(faces) != 0:118 # 矩形框 / Show the ROI of faces119 for k, d in enumerate(faces):120 # 计算矩形框大小 / Compute the size of rectangle box121 height = (d.bottom() - d.top())122 width = (d.right() - d.left())123 hh = int(height/2)124 ww = int(width/2)125126 # 6. 判断人脸矩形框是否超出 480x640 / If the size of ROI > 480x640127 if (d.right()+ww) > 640 or (d.bottom()+hh > 480) or (d.left()-ww < 0) or (d.top()-hh < 0):128 cv2.putText(img_rd, "OUT OF RANGE", (20, 300), self.font, 0.8, (0, 0, 255), 1, cv2.LINE_AA)129 color_rectangle = (0, 0, 255)130 save_flag = 0131 if kk == ord('s'):132 print("请调整位置 / Please adjust your position")133 else:134 color_rectangle = (255, 255, 255)135 save_flag = 1136137 cv2.rectangle(img_rd,138 tuple([d.left() - ww, d.top() - hh]),139 tuple([d.right() + ww, d.bottom() + hh]),140 color_rectangle, 2)141142 # 7. 根据人脸大小生成空的图像 / Create blank image according to the size of face detected143 img_blank = np.zeros((int(height*2), width*2, 3), np.uint8)144145 if save_flag:146 # 8. 按下 's' 保存摄像头中的人脸到本地 / Press 's' to save faces into local images147 if kk == ord('s'):148 # 检查有没有先按'n'新建文件夹 / Check if you have pressed 'n'149 if self.press_n_flag:150 self.ss_cnt += 1151 for ii in range(height*2):152 for jj in range(width*2):153 img_blank[ii][jj] = img_rd[d.top()-hh + ii][d.left()-ww + jj]154 cv2.imwrite(current_face_dir + "/img_face_" + str(self.ss_cnt) + ".jpg", img_blank)155 print("写入本地 / Save into：", str(current_face_dir) + "/img_face_" + str(self.ss_cnt) + ".jpg")156 else:157 print("请先按 'N' 来建文件夹, 按 'S' / Please press 'N' and press 'S'")158159 self.current_frame_faces_cnt = len(faces)160161 # 9. 生成的窗口添加说明文字 / Add note on cv2 window162 self.draw_note(img_rd)163164 # 10. 按下 'q' 键退出 / Press 'q' to exit165 if kk == ord('q'):166 break167168 # 11. Update FPS169 self.update_fps()170171 cv2.namedWindow("camera", 1)172 cv2.imshow("camera", img_rd)173174 def run(self):175 cap = cv2.VideoCapture(0)176 self.process(cap)177178 cap.release()179 cv2.destroyAllWindows()180181182 def main():183 Face_Register_con = Face_Register()184 Face_Register_con.run()185186187 if __name__ == '__main__':188 main()
考虑到有可能需要保存的矩形框超出摄像头范围，对于这种异常，如果矩形框超出范围，矩形框会从白变红，然后提示 "OUT OF RANGE"，这时候不允许用户保存人脸图片;
图 5 人脸录入异常（Out of range）处理
get_face_from_camera.py 的输出 log：
新建的人脸文件夹 / Create folders: data/data_faces_from_camera/person_1写入本地 / Save into： data/data_faces_from_camera/person_1/img_face_1.jpg写入本地 / Save into： data/data_faces_from_camera/person_1/img_face_2.jpg写入本地 / Save into： data/data_faces_from_camera/person_1/img_face_3.jpg写入本地 / Save into： data/data_faces_from_camera/person_1/img_face_4.jpg新建的人脸文件夹 / Create folders: data/data_faces_from_camera/person_2写入本地 / Save into： data/data_faces_from_camera/person_2/img_face_1.jpg写入本地 / Save into： data/data_faces_from_camera/person_2/img_face_2.jpg新建的人脸文件夹 / Create folders: data/data_faces_from_camera/person_3写入本地 / Save into： data/data_faces_from_camera/person_3/img_face_1.jpg写入本地 / Save into： data/data_faces_from_camera/person_3/img_face_2.jpg
2.1.2 GUI with Tkinter
利用 Tkinter 生成的 GUI，用户界面更加丰富，也可以在该界面，录入人脸的时候设置姓名；
点击 Step 3 中的按钮“Save current face”，将当前帧中检测到的人脸保存下来；
2.2 features_extraction_to_csv.py
这部分代码实现的功能是将之前捕获到的人脸图像文件，提取出 128D 特征，然后计算出某人人脸数据的特征均值存入 CSV 中，方便之后识别时候进行比对；
利用 numpy.mean() 计算特征均值，生成一个存储所有录入人脸数据 database 的 "features_all.csv"；
源码:
1 # Copyright (C) 2018-2021 coneypo 2 # SPDX-License-Identifier: MIT 3 4 # Author: coneypo 5 # Blog: 6 # GitHub: 7 # Mail: coneypo@foxmail.com 8 9 # 从人脸图像文件中提取人脸特征存入 "features_all.csv" / Extract features from images and save into "features_all.csv" 10 11 import os 12 import dlib 13 import csv 14 import numpy as np 15 import logging 16 import cv2 17 18 # 要读取人脸图像文件的路径 / Path of cropped faces 19 path_images_from_camera = "data/data_faces_from_camera/" 20 21 # Dlib 正向人脸检测器 / Use frontal face detector of Dlib 22 detector = dlib.get_frontal_face_detector() 23 24 # Dlib 人脸 landmark 特征点检测器 / Get face landmarks 25 predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat') 26 27 # Dlib Resnet 人脸识别模型，提取 128D 的特征矢量 / Use Dlib resnet50 model to get 128D face descriptor 28 face_reco_model = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat") 29 30 31 # 返回单张图像的 128D 特征 / Return 128D features for single image 32 # Input: path_img33 # Output: face_descriptor 34 def return_128d_features(path_img): 35 img_rd = cv2.imread(path_img) 36 faces = detector(img_rd, 1) 37 38 logging.info("%-40s %-20s", "检测到人脸的图像 / Image with faces detected:", path_img) 39 40 # 因为有可能截下来的人脸再去检测，检测不出来人脸了, 所以要确保是 检测到人脸的人脸图像拿去算特征 41 # For photos of faces saved, we need to make sure that we can detect faces from the cropped images 42 if len(faces) != 0: 43 shape = predictor(img_rd, faces[0]) 44 face_descriptor = face_reco_model.compute_face_descriptor(img_rd, shape) 45 else: 46 face_descriptor = 0 47 logging.warning("no face") 48 return face_descriptor 49 50 51 # 返回 personX 的 128D 特征均值 / Return the mean value of 128D face descriptor for person X 52 # Input: path_face_personX 53 # Output: features_mean_personX 54 def return_features_mean_personX(path_face_personX): 55 features_list_personX = [] 56 photos_list = os.listdir(path_face_personX) 57 if photos_list: 58 for i in range(len(photos_list)): 59 # 调用 return_128d_features() 得到 128D 特征 / Get 128D features for single image of personX 60 logging.info("%-40s %-20s", "正在读的人脸图像 / Reading image:", path_face_personX + "/" + photos_list[i]) 61 features_128d = return_128d_features(path_face_personX + "/" + photos_list[i]) 62 # 遇到没有检测出人脸的图片跳过 / Jump if no face detected from image 63 if features_128d == 0: 64 i += 1 65 else: 66 features_list_personX.append(features_128d) 67 else: 68 logging.warning("文件夹内图像文件为空 / Warning: No images in%s/", path_face_personX) 69 70 # 计算 128D 特征的均值 / Compute the mean 71 # personX 的 N 张图像 x 128D -> 1 x 128D 72 if features_list_personX: 73 features_mean_personX = np.array(features_list_personX, dtype=object).mean(axis=0) 74 else: 75 features_mean_personX = np.zeros(128, dtype=int, order='C') 76 return features_mean_personX 77 78 79 def main(): 80 logging.basicConfig(level=logging.INFO) 81 # 获取已录入的最后一个人脸序号 / Get the order of latest person 82 person_list = os.listdir("data/data_faces_from_camera/") 83 person_list.sort() 84 85 with open("data/features_all.csv", "w", newline="") as csvfile: 86 writer = csv.writer(csvfile) 87 for person in person_list: 88 # Get the mean/average features of face/personX, it will be a list with a length of 128D 89 logging.info("%sperson_%s", path_images_from_camera, person) 90 features_mean_personX = return_features_mean_personX(path_images_from_camera + person) 91 92 if len(person.split('_', 2)) == 2: 93 # "person_x" 94 person_name = person 95 else: 96 # "person_x_tom" 97 person_name = person.split('_', 2)[-1] 98 features_mean_personX = np.insert(features_mean_personX, 0, person_name, axis=0) 99 # features_mean_personX will be 129D, person name + 128 features100 writer.writerow(features_mean_personX)101 logging.info('\n')102 logging.info("所有录入人脸数据存入 / Save all the features of faces registered into: data/features_all.csv")103104105 if __name__ == '__main__':106 main()
我们可以看下对于某张图片，face_descriptor 这个 128D vectors 的输出结果：
绿色框内是我们的返回 128D 特征的函数；
在红色框内调用该函数来计算 img_face_13.jpg；
可以看到黄色框中的输出为 128D 的向量；
图 6 返回单张图像的 128D 特征的计算结果
之后就需要人脸图像进行批量化操作，提取出 128D 的特征，然后计算特征均值，存入 features_all.csv；
features_all.csv 是一个 n 行 129 列的 CSV， n 是录入的人脸数，129 列是某人的名字加上 128D 特征（如果没有设置名字，那么就是 person_1, person_2 之类）；
这存储的就是 录入的人脸特征数据，之后 摄像头捕获的人脸 将要拿过来和 这些特征值 进行比对，如果欧式距离比较近的话，就可以认为是同一张人脸；
get_features_into_CSV.py 的输出 log：
##### person_1 #####data/data_csvs_from_camera/person_1.csv正在读的人脸图像 / image to read: data/data_faces_from_camera/person_1/img_face_1.jpg检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_1/img_face_1.jpg正在读的人脸图像 / image to read: data/data_faces_from_camera/person_1/img_face_2.jpg检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_1/img_face_2.jpg正在读的人脸图像 / image to read: data/data_faces_from_camera/person_1/img_face_3.jpg检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_1/img_face_3.jpg正在读的人脸图像 / image to read: data/data_faces_from_camera/person_1/img_face_4.jpg检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_1/img_face_4.jpg##### person_2 #####data/data_csvs_from_camera/person_2.csv正在读的人脸图像 / image to read: data/data_faces_from_camera/person_2/img_face_1.jpg检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_2/img_face_1.jpg正在读的人脸图像 / image to read: data/data_faces_from_camera/person_2/img_face_2.jpg检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_2/img_face_2.jpg##### person_3 #####data/data_csvs_from_camera/person_3.csv正在读的人脸图像 / image to read: data/data_faces_from_camera/person_3/img_face_1.jpg检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_3/img_face_1.jpg正在读的人脸图像 / image to read: data/data_faces_from_camera/person_3/img_face_2.jpg检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_3/img_face_2.jpg...
2.3 face_reco_from_camera.py / 实时人脸识别对比分析
这部分源码实现的功能：调用摄像头，捕获摄像头中的人脸，然后如果检测到人脸，将 摄像头中的人脸提取出 128D 的特征，然后和 之前录入人脸的 128D 特征 进行计算欧式距离，如果比较小，可以判定为一个人，否则不是一个人；
图 7 face_reco_from_camera.py 实现逻辑
所以设计的伪代码如下：
# 人脸检测器/预测器/识别模型detector = dlib.get_frontal_face_detector()predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat')facerec = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat")faces = detector(img_gray, 0)# 1. 如果检测到人脸if len(faces) != 0: # 遍历所有检测到的人脸 for i in range(len(faces)): # 2. 提取当前帧人脸的特征描述子 shape = predictor(img_rd, faces[i]) facerec.compute_face_descriptor(img_rd, shape) # 3. 将当前帧人脸特征描述子和数据库的特征描述子进行对比 for i in range(len(self.features_known_list)): e_distance_tmp = self.return_euclidean_distance(self.features_camera_list[k], self.features_known_list[i])
关于 里面变量的定义：
关于用到的 dlib 检测器，预测器，识别器:
1. dlib.get_frontal_face_detector
Link：
介绍：返回默认的人脸检测器，为下面的 fhog_object_detectorm / Returns the default face detector
2. class dlib.fhog_object_detector
Link：
介绍 :基于滑动窗的HOG进行目标检测, This object represents a sliding window histogram-of-oriented-gradients based object detector.
参数:
__call__(self: dlib.fhog_object_detector, image: array, upsample_num_times: int=0L) → dlib.rectangles
3. class dlib.shape_predictor
Link:
简介:
人脸图像作为输入, 输出面部特征点;
This object is a tool that takes in an image region containing some object and outputs a set of point locations that define the pose of the object.
The classic example of this is human face pose prediction, where you take an image of a human face as input and are expected to identify
the locations of important facial landmarks such as the corners of the mouth and eyes, tip of the nose, and so forth.
参数 / parameters:
__call__(self: dlib.shape_predictor, image: array, box: dlib.rectangle) → dlib.full_object_detection输入: dlib.rectangle 输出: dlib.full_object_detection
4. class dlib.face_recognition_model_v1
Link:
简介 / intro:
将人脸转换为128D特征向量, 这样的话相似人脸会比较相近, 不相像的会比较远;
This object maps human faces into 128D vectors where pictures of the same person are mapped near to each other and pictures of different people are mapped far apart.
The constructor loads the face recognition model from a file. The model file is available here:
参数 / parameters:
compute_face_descriptor(self: dlib.face_recognition_model_v1, img: numpy.ndarray[(rows,cols,3),uint8], face: dlib.full_object_detection, num_jitters: int=0L, padding: float=0.25) -> dlib.vector
通过 print(type()) 可以更清楚的看到 dlib 对象的传递:
# 人脸检测器/预测器/识别模型detector = dlib.get_frontal_face_detector()predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat')facerec = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat")faces = detector(img_gray, 0)# 如果检测到人脸if len(faces) != 0: print(type(faces) #
这样一个对象传递过程：
faces = detector(img_gray, 0) -> <class 'dlib.dlib.rectangles'> ->shape = predictor(img_rd, faces[i]) -> <class 'dlib.dlib.full_object_detection'> ->facerec.compute_face_descriptor(img_rd, shape) -> <class 'dlib.dlib.vector'>
欧氏距离对比的阈值设定，是在 return_euclidean_distance 函数的 dist 变量；
我这里程序里面指定的 欧氏距离判断阈值是 0.4，具体阈值可以根据实际情况或者测得结果进行修改；
这边做了一个，让人名跟随显示在头像下方，如果想要在人脸矩形框下方显示人名，首先需要知道 Dlib 生成的矩形框的尺寸怎么读取；
Dlib 返回的 dets 变量是一系列人脸的数据，此处对单张人脸处理，所以取 dets[0] 的参数；
可以通过 dets[0].top(), dets[0].bottom(), dets[0].left() 和 dets[0].right() 来确定要显示的人名的坐标；
图 8 dets[0].top() 等参数说明
得到矩形框的坐标，就可以获取人名的相对位置；
这是我这边取的坐标：
pos_text_1 = tuple([dets[0].left(), int(dets[0].bottom()+(dets[0].bottom()-dets[0].top())/4)])
图 9 face_reco_from_camera.py 生成的人脸识别窗口界面
如果想定制输出显示的名字而不是“Person_1”,"Person_2"...；
图 9 定制显示名字
源码：
1 # Copyright (C) 2020 coneypo 2 # SPDX-License-Identifier: MIT 3 4 # Author: coneypo 5 # Blog: 6 # GitHub: 7 # Mail: coneypo@foxmail.com 8 9 # 摄像头实时人脸识别 / Real-time face detection and recognition 10 11 import dlib 12 import numpy as np 13 import cv2 14 import pandas as pd 15 import os 16 import time 17 from PIL import Image, ImageDraw, ImageFont 18 19 # Dlib 正向人脸检测器 / Use frontal face detector of Dlib 20 detector = dlib.get_frontal_face_detector() 21 22 # Dlib 人脸 landmark 特征点检测器 / Get face landmarks 23 predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat') 24 25 # Dlib Resnet 人脸识别模型，提取 128D 的特征矢量 / Use Dlib resnet50 model to get 128D face descriptor 26 face_reco_model = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat") 27 28 29 class Face_Recognizer: 30 def __init__(self): 31 self.feature_known_list = [] # 用来存放所有录入人脸特征的数组 / Save the features of faces in the database 32 self.name_known_list = [] # 存储录入人脸名字 / Save the name of faces in the database 33 34 self.current_frame_face_cnt = 0 # 存储当前摄像头中捕获到的人脸数 / Counter for faces in current frame 35 self.current_frame_feature_list = [] # 存储当前摄像头中捕获到的人脸特征 / Features of faces in current frame 36 self.current_frame_name_position_list = [] # 存储当前摄像头中捕获到的所有人脸的名字坐标 / Positions of faces in current frame 37 self.current_frame_name_list = [] # 存储当前摄像头中捕获到的所有人脸的名字 / Names of faces in current frame 38 39 # Update FPS 40 self.fps = 0 41 self.frame_start_time = 0 42 43 # 从 "features_all.csv" 读取录入人脸特征 / Get known faces from "features_all.csv" 44 def get_face_database(self): 45 if os.path.exists("data/features_all.csv"): 46 path_features_known_csv = "data/features_all.csv" 47 csv_rd = pd.read_csv(path_features_known_csv, header=None) 48 for i in range(csv_rd.shape[0]): 49 features_someone_arr = [] 50 for j in range(0, 128): 51 if csv_rd.iloc[i][j] == '': 52 features_someone_arr.append('0') 53 else: 54 features_someone_arr.append(csv_rd.iloc[i][j]) 55 self.feature_known_list.append(features_someone_arr) 56 self.name_known_list.append("Person_"+str(i+1)) 57 print("Faces in Database：", len(self.feature_known_list)) 58 return 1 59 else: 60 print('##### Warning #####', '\n') 61 print("'features_all.csv' not found!") 62 print( 63 "Please run 'get_faces_from_camera.py' and 'features_extraction_to_csv.py' before 'face_reco_from_camera.py'", 64 '\n') 65 print('##### End Warning #####') 66 return 0 67 68 # 计算两个128D向量间的欧式距离 / Compute the e-distance between two 128D features 69 @staticmethod 70 def return_euclidean_distance(feature_1, feature_2): 71 feature_1 = np.array(feature_1) 72 feature_2 = np.array(feature_2) 73 dist = np.sqrt(np.sum(np.square(feature_1 - feature_2))) 74 return dist 75 76 # 更新 FPS / Update FPS of Video stream 77 def update_fps(self): 78 now = time.time() 79 self.frame_time = now - self.frame_start_time 80 self.fps = 1.0 / self.frame_time 81 self.frame_start_time = now 82 83 def draw_note(self, img_rd): 84 font = cv2.FONT_ITALIC 85 86 cv2.putText(img_rd, "Face Recognizer", (20, 40), font, 1, (255, 255, 255), 1, cv2.LINE_AA) 87 cv2.putText(img_rd, "FPS: " + str(self.fps.__round__(2)), (20, 100), font, 0.8, (0, 255, 0), 1, cv2.LINE_AA) 88 cv2.putText(img_rd, "Faces: " + str(self.current_frame_face_cnt), (20, 140), font, 0.8, (0, 255, 0), 1, cv2.LINE_AA) 89 cv2.putText(img_rd, "Q: Quit", (20, 450), font, 0.8, (255, 255, 255), 1, cv2.LINE_AA) 90 91 def draw_name(self, img_rd): 92 # 在人脸框下面写人脸名字 / Write names under rectangle 93 font = ImageFont.truetype("simsun.ttc", 30) 94 img = Image.fromarray(cv2.cvtColor(img_rd, cv2.COLOR_BGR2RGB)) 95 draw = ImageDraw.Draw(img) 96 for i in range(self.current_frame_face_cnt): 97 # cv2.putText(img_rd, self.current_frame_name_list[i], self.current_frame_name_position_list[i], font, 0.8, (0, 255, 255), 1, cv2.LINE_AA) 98 draw.text(xy=self.current_frame_name_position_list[i], text=self.current_frame_name_list[i], font=font) 99 img_with_name = cv2.cvtColor(np.array(img), cv2.COLOR_RGB2BGR)100 return img_with_name101102 # 修改显示人名 / Show names in chinese103 def show_chinese_name(self):104 # Default known name: person_1, person_2, person_3105 if self.current_frame_face_cnt >= 1:106 self.name_known_list[0] ='张三'.encode('utf-8').decode()107 # self.name_known_list[1] ='李四'.encode('utf-8').decode()108 # self.name_known_list[2] ='xx'.encode('utf-8').decode()109 # self.name_known_list[3] ='xx'.encode('utf-8').decode()110 # self.name_known_list[4] ='xx'.encode('utf-8').decode()111112 # 处理获取的视频流，进行人脸识别 / Face detection and recognition from input video stream113 def process(self, stream):114 # 1. 读取存放所有人脸特征的 csv / Get faces known from "features.all.csv"115 if self.get_face_database():116 while stream.isOpened():117 print(">>> Frame start")118 flag, img_rd = stream.read()119 faces = detector(img_rd, 0)120 kk = cv2.waitKey(1)121 # 按下 q 键退出 / Press 'q' to quit122 if kk == ord('q'):123 break124 else:125 self.draw_note(img_rd)126 self.current_frame_feature_list = []127 self.current_frame_face_cnt = 0128 self.current_frame_name_position_list = []129 self.current_frame_name_list = []130131 # 2. 检测到人脸 / Face detected in current frame132 if len(faces) != 0:133 # 3. 获取当前捕获到的图像的所有人脸的特征 / Compute the face descriptors for faces in current frame134 for i in range(len(faces)):135 shape = predictor(img_rd, faces[i])136 self.current_frame_feature_list.append(face_reco_model.compute_face_descriptor(img_rd, shape))137 # 4. 遍历捕获到的图像中所有的人脸 / Traversal all the faces in the database138 for k in range(len(faces)):139 print(">>>>>> For face", k+1, " in camera")140 # 先默认所有人不认识，是 unknown / Set the default names of faces with "unknown"141 self.current_frame_name_list.append("unknown")142143 # 每个捕获人脸的名字坐标 / Positions of faces captured144 self.current_frame_name_position_list.append(tuple(145 [faces[k].left(), int(faces[k].bottom() + (faces[k].bottom() - faces[k].top()) / 4)]))146147 # 5. 对于某张人脸，遍历所有存储的人脸特征148 # For every faces detected, compare the faces in the database149 current_frame_e_distance_list = []150 for i in range(len(self.feature_known_list)):151 # 如果 person_X 数据不为空152 if str(self.feature_known_list[i][0]) != '0.0':153 print(" >>> With person", str(i + 1), ", the e distance: ", end='')154 e_distance_tmp = self.return_euclidean_distance(self.current_frame_feature_list[k],155 self.feature_known_list[i])156 print(e_distance_tmp)157 current_frame_e_distance_list.append(e_distance_tmp)158 else:159 # 空数据 person_X160 current_frame_e_distance_list.append(999999999)161 # 6. 寻找出最小的欧式距离匹配 / Find the one with minimum e distance162 similar_person_num = current_frame_e_distance_list.index(min(current_frame_e_distance_list))163 print(" >>> Minimum e distance with ", self.name_known_list[similar_person_num], ": ", min(current_frame_e_distance_list))164165 if min(current_frame_e_distance_list) < 0.4:166 self.current_frame_name_list[k] = self.name_known_list[similar_person_num]167 print(" >>> Face recognition result: " + str(self.name_known_list[similar_person_num]))168 else:169 print(" >>> Face recognition result: Unknown person")170171 # 矩形框 / Draw rectangle172 for kk, d in enumerate(faces):173 # 绘制矩形框174 cv2.rectangle(img_rd, tuple([d.left(), d.top()]), tuple([d.right(), d.bottom()]),175 (0, 255, 255), 2)176177 self.current_frame_face_cnt = len(faces)178179 # 7. 在这里更改显示的人名 / Modify name if needed180 # self.show_chinese_name()181182 # 8. 写名字 / Draw name183 img_with_name = self.draw_name(img_rd)184185 else:186 img_with_name = img_rd187188 print(">>>>>> Faces in camera now:", self.current_frame_name_list)189190 cv2.imshow("camera", img_with_name)191192 # 9. 更新 FPS / Update stream FPS193 self.update_fps()194 print(">>> Frame ends\n\n")195196 # OpenCV 调用摄像头并进行 process197 def run(self):198 cap = cv2.VideoCapture(0)199 # cap = cv2.VideoCapture("video.mp4")200 cap.set(3, 480) # 640x480201 self.process(cap)202203 cap.release()204 cv2.destroyAllWindows()205206207 def main():208 Face_Recognizer_con = Face_Recognizer()209 Face_Recognizer_con.run()210211212 if __name__ == '__main__':213 main()
face_reco_from_camera.py 输出 log：
Faces in Database： 3>>> Frame start>>>>>> For face 1 in camera >>> With person 2 , the e distance: 0.24747225595381367 >>> With person 3 , the e distance: 0.22821104803792178 >>> Minimum e distance with Person_3 : 0.22821104803792178 >>> Face recognition result: Person_3>>>>>> Faces in camera now: ['Person_3']>>> Frame ends>>> Frame start>>>>>> For face 1 in camera >>> With person 2 , the e distance: 0.2490812900317618 >>> With person 3 , the e distance: 0.22549497689337802 >>> Minimum e distance with Person_3 : 0.22549497689337802 >>> Face recognition result: Person_3>>>>>> Faces in camera now: ['Person_3']>>> Frame ends>>> Frame start>>>>>> For face 1 in camera >>> With person 2 , the e distance: 0.24569769385882426 >>> With person 3 , the e distance: 0.2262102554355137 >>> Minimum e distance with Person_3 : 0.2262102554355137 >>> Face recognition result: Person_3>>>>>> Faces in camera now: ['Person_3']>>> Frame ends>>> Frame start>>>>>> For face 1 in camera >>> With person 2 , the e distance: 0.24387949251367172 >>> With person 3 , the e distance: 0.22636200199905795 >>> Minimum e distance with Person_3 : 0.22636200199905795 >>> Face recognition result: Person_3>>>>>> Faces in camera now: ['Person_3']>>> Frame ends>>> Frame start>>>>>> For face 1 in camera >>> With person 2 , the e distance: 0.2473446948271673 >>> With person 3 , the e distance: 0.22534075942468246 >>> Minimum e distance with Person_3 : 0.22534075942468246 >>> Face recognition result: Person_3>>>>>> Faces in camera now: ['Person_3']>>> Frame ends>>> Frame start>>>>>> For face 1 in camera >>> With person 2 , the e distance: 0.24465000646050672 >>> With person 3 , the e distance: 0.2238005841538998 >>> Minimum e distance with Person_3 : 0.2238005841538998 >>> Face recognition result: Person_3>>>>>> Faces in camera now: ['Person_3']>>> Frame ends
如果对单个人脸，进行实时对比输出：
图 10 实时输出的欧氏距离结果
通过实时的输出结果，看的比较明显；
输出绿色部分：当是我自己时，计算出来的欧式距离基本都在 0.2 左右；
输出红色部分：而换一张图片上去比如特朗普，明显看到欧式距离计算结果 达到了 0.8，此时就可以判定，后来这张人脸不是一张人脸；
所以之前提到的欧式距离计算对比的阈值可以由此设定，本项目中取的是 dist=0.4；
dist 的确切取值自己权衡， 的说明：
# When using a distance threshold of 0.6, the dlib model obtains an accuracy# of 99.38% on the standard LFW face recognition benchmark, which is# comparable to other state-of-the-art methods for face recognition as of# February 2017. This accuracy means that, when presented with a pair of face# images, the tool will correctly identify if the pair belongs to the same# person or is from different people 99.38% of the time.
可以通过 logging.basicConfig(level=logging.DEBUG) 来设置代码的打印；3. 总结
核心就是 提取人脸特征，然后计算欧式距离和预设的特征脸进行比对；
不过这个实时获取摄像头人脸进行比对，要实时的进行计算摄像头脸的特征值，然后还要计算欧氏距离，所以计算量比较大，可能摄像头视频流会出现卡顿；
此项目仅个人学习爱好研究，开源供大家一起学习；
# 请尊重他人劳动成果，转载或者使用源码请注明出处：
# 代码已上传到了我的 GitHub，如果对您有帮助欢迎 Star 支持我下：
# 如有问题请留言或者联系邮箱:
本文地址：https://caijingdemo.com/changshi/75750.html
