Direct Shape Regression Networks for End-to-End Face Alignment

Direct Shape Regression Networks for End-to-End Face Alignment

Face alignment has been extensively studied in computer vision community due to its fundamental role in facial analysis, but it remains an unsolved problem. The major challenges lie in the highly nonlinear relationship between face images and associated facial shapes, which is coupled by underlying correlation of landmarks.Existing methods mainly rely on cascaded regression, suffering from intrinsic shortcomings, e.g., strong dependency on initialization and failure to exploit landmark correlations. In this paper, we propose the direct shape regression network (DSRN) for end-to-end face alignment by jointly handling the aforementioned challenges in a unified framework. Specifically, by deploying doubly convolutional layer and by using the Fourier feature pooling layer proposed in this paper, DSRN efficiently constructs strong representations to disentangle highly nonlinear relationships between images and shapes; by incorporating a linear layer of low-rank learning, DSRN effectively encodes correlations of landmarks to improve performance. DSRN leverages the strengths of kernels for nonlinear feature extraction and neural networks for structured prediction, and provides the first end-to-end learning architecture for direct face alignment. Its effectiveness and generality are validated by extensive experiments on five benchmark datasets, including AFLW, 300W, CelebA, MAFL, and 300VW. All empirical results demonstrate that DSRN consistently produces high performance and in most cases surpasses state-of-the-art.

直接形状回归网络用于端对端的面对齐

由于面部对齐技术在面部分析中的基本作用，因此已在计算机视觉社区中进行了广泛的研究，但它仍然是一个尚未解决的问题。主要的挑战在于人脸图像与相关的人脸形状之间的高度非线性关系，这与地标的潜在相关性相关。.. 现有的方法主要依靠级联回归，存在固有的缺点，例如，强烈依赖于初始化以及无法利用界标相关性。在本文中，我们通过在统一框架中共同应对上述挑战，提出了用于端到端面部对齐的直接形状回归网络（DSRN）。具体而言，通过部署双卷积层并使用本文提出的傅立叶特征池层，DSRN有效地构造了强大的表示形式，以解开图像与形状之间的高度非线性关系。通过合并低秩学习的线性层，DSRN有效地编码了地标的相关性以提高性能。DSRN利用内核的优势进行非线性特征提取，并利用神经网络进行结构化预测，并提供了第一个用于直接人脸对齐的端到端学习架构。通过对五个基准数据集（包括AFLW，300W，CelebA，MAFL和300VW）进行广泛的实验，验证了其有效性和通用性。所有的经验结果表明，DSRN始终能够产生高性能，并且在大多数情况下都超过了最新技术。 （阅读更多）