对抗训练在NLP中的应用

标签：对抗训练

这篇论文将图像领域的adversarial training 应用在了文本领域，改善了word embedding.传统的embedding被语法结构影响，可以对两个分布式相同，共现关系相同但是意义完全相反的词（比如，good和bad）分开，对于情感分类问题有很好的效果。

关于论文

ICLR 2017 Adversarial Training Methods for Semi-Supervised Text Classification
Deep mind

对抗样本

首先看看对抗样本、对抗训练、虚拟对抗训练的定义和特点

Adversarial examples are examples that are created by making small perturbations to the input designed to significantly increase the loss incurred by a machine learning model (Szegedy et al., 2014; Goodfellow et al., 2015)

对抗样本：通过对输入制造很小的扰动，模型的loss显著性增加

Adversarial training requires the use of labels when training models that use a supervised cost, because the label appears in the cost function that the adversarial perturbation is designed to maximize.

对抗训练: 需要label,因为其损失函数中有label的信息，因此输入监督学习的范畴
Virtual adversarial training (Miyato et al., 2016) extends the idea of adversarial training to the semi-supervised regime and unlabeled examples. This is done by regularizing themodel so that given an example, the model will produce the same output distribution as it produces on an adversarial perturbation of that example. Virtual adversarial training achieves good generalization performance for both supervised and semi-supervised learning tasks.
Previous
virtual 对抗训练 是对对抗训练的扩展，可以用到半监督学习的领域中，主要在于其损失函数，下面会有介绍。