根据关键点对齐面部区域

对齐图像：
https://cloud.tencent.com/developer/article/1536449

对齐后选择边界模式：
在warpAffine变化之后，空余区域默认是用constant方式填充的。
如果不选择的话默认是用黑色填充，可能对网络训练产生影响。

① cv2.BORDER_TRANSPARENT: uvwxyz|absdefgh|ijklmno

image.png

② cv2.BORDER_WRAP cdefgh|abcdefgh|abcdefg

image.png

③ cv2.BORDER_REPLICATE

aaaaaa|abcdefgh|hhhhhhh

image.png image.png

apply rotation matrix to 2D-points
https://www.science-emergence.com/Articles/How-to-create-and-apply-a-rotation-matrix-using-python-/
https://zhuanlan.zhihu.com/p/55479744

image.png

代码：

    def _align_eyes(self, img, eye_ldmks, img_shape):
        """
        Align expanded eye roi using reference eye location point.
        img:
            Eye roi image
        eye_location:
            Two 2-D point, representing left/right eyelid center
        return:
            Aligned eye roi.
        """
        assert(len(eye_ldmks.shape)>1), \
            "Param eye_ldmks must be 2D array, given: {}".format(eye_ldmks.shape)
        left_center_ref, right_center_ref = self.reference_eye_location * img_shape

        eye_ldmks = eye_ldmks * img_shape
        left_center = np.mean(eye_ldmks[0:8], axis=0)
        right_center = np.mean(eye_ldmks[16:24], axis=0)

        delta_x = right_center[0] - left_center[0]
        delta_y = right_center[1] - left_center[1]

        dist_ref = right_center_ref[0] - left_center_ref[0]
        dist = np.sqrt((delta_x**2) + (delta_y**2))

        # 各向同性缩放因子
        scale = dist_ref / dist
        # 旋转角度
        angle = np.degrees(np.arctan2(delta_y, delta_x))
        # 旋转中心
        center = (
            int(left_center[0]+right_center[0])/2,
            int(left_center[1]+right_center[1])/2
        )
        # 计算旋转矩阵
        M = cv2.getRotationMatrix2D(center, angle, scale)
        # 计算x,y方向平移
        M[0, 2] += self._resize_hw / 2 - center[0]
        M[1, 2] += left_center_ref[1] - center[1]

        # 进行仿射变换
        eye_roi_aligned = cv2.warpAffine(
            img, M, (img_shape[1], img_shape[0]), borderMode=cv2.BORDER_REPLICATE #flags=cv2.INTER_CUBIC
        )
        # 对ldmk进行变换
        eye_ldmks = np.c_[eye_ldmks, np.ones((eye_ldmks.shape[0]))]
        ldmks_aligned = M.dot(eye_ldmks.T).T / img_shape  # normalize to 0-1

        return eye_roi_aligned, ldmks_aligned.reshape(-1) # , M # center, angle, scale