Problem:

Given the profile of a person(text format), how to build an open-domain chat-bot with personality?

open-domain chat-bot means a chat-bot with large knowledge bases;
personality means:
- personalized speaking style
- response profile correctly

For response profile correctly, the work is done by [Qiao et al., 2018] for some simple attributes like age, gender; Here is an example:

image_1cp9v6rpi15p9173k6re22fcrum.png-81.3kB

But it still needs more complex semantic reasoning like when this chat-bot is asked 'Are you married?'(Assume the answer to this question is beyond the profile and could be reasoned by age(3 years old).)

And for personalized speaking style, the object is to make chat-bot generate responses with specific speaking style; It is difficult to formalize the style, and also the metric is not so clear. So, it also needs manual evaluation.
But to make a chat-bot like a human, it is a promising work.

Metrics：

Auto metrics
- BLEU
- ROUGE
- METEOR

An example shows the shortcomings of BLEU:

An example shows the shortcomings of BLEU

We ﬁnd that all metrics show either weak or no correlation with human judgements, despite the fact that word overlap metrics have been used extensively in the literature for evaluating dialogue response models.

---[Chia-Wei et al., 2016]

So, the work of creating a better metric to measure the generated conversations is also needed.

close domain的问题里，有一个ubuntu的数据集和评测方法：
https://arxiv.org/abs/1506.08909