A survey on contrastive self-supervised learning
https://arxiv.org/abs/2011.00362
Self-supervised learning has gained popularity because of its ability to avoid the cost of annotating large-scale datasets. It is capable of adopting self-defined pseudo labels as supervision and use the learned representations for several downstream tasks. Specifically, contrastive learning has recently become a dominant component in self-supervised learning methods for computer vision, natural language processing (NLP), and other domains. It aims at embedding augmented versions of the same sample close to each other while trying to push away embeddings from different samples. This paper provides an extensive review of self-supervised methods that follow the contrastive approach. The work explains commonly used pretext tasks in a contrastive learning setup, followed by different architectures that have been proposed so far. Next, we have a performance comparison of different methods for multiple downstream tasks such as image classification, object detection, and action recognition. Finally, we conclude with the limitations of the current methods and the need for further techniques and future directions to make substantial progress
自监督学习因其能够避免注释大规模数据集的成本而广受欢迎。它被定义为可以使用多个下游任务的自学习和伪表示。具体来说，对比学习已经成为计算机视觉、自然语言处理等领域的自监督学习方法的主要组成部分。它的目标是嵌入同一个样本的增强版本，使它们彼此靠近，同时试图从不同的样本中推开嵌入。本文提供了一个广泛的回顾自我监督的方法，遵循对比法。这项工作解释了在对比学习中常用的借口任务，以及到目前为止提出的不同结构。接下来，我们对多个下游任务（如图像分类、目标检测和动作识别）的不同方法进行了性能比较。最后，我们总结了现有方法的局限性，以及需要进一步的技术和未来的方向来取得实质性进展
网友评论