HoloLens开发手记 - 语音识别（语音命令）

语音表达是人类最直接的表达方式之一，语音输入不像手势识别的AirTap操作久了会让用户觉得手部酸痛。对于HoloLens，语音输入是三大基本输入方式之一，广泛地运用在各种交互场所上。Hololens上语音输入有三种形式，分别是：

• 语音命令 Voice Command
• 听写 Diction
• 语法识别 Grammar Recognizer

本文主要介绍语音命令，对于HoloLens的交互使用来说，语音命令也是最经常被使用的语音输入形式。

语音命令 Voice Command

---

使用语音命令，首先的确保在应用中已经开启 microphone 的功能特性，具体设置是 Edit -> Project Settings -> Player -> Settings for Windows Store -> Publishing Settings -> Capabilities 下找到 microphone 特性并勾上。

对于语音命令的使用，开发者通过声明 Dictionary<string, UnityEvent> 来设定语音命令的关键字和对应行为，并由这个注册 Dictionary 来初始化 KeywordRecognizer ，编写行为动作函数。当用户说出关键词时，预设的动作就会被调用，从而实现语音命令的功能。

**KeywordRecognizer.cs**

using UnityEngine.Windows.Speech;
using System.Collections.Generic;
using System.Linq;

public class KeywordManager : MonoBehavior
{

    KeywordRecognizer keywordRecognizer;
    Dictionary<string, System.Action> keywords = new Dictionary<string, System.Action>();

    void Start()
    {
        //初始化关键词
        keywords.Add("activate", () =>
        {
            // 想执行的行为
        });


        keywordRecognizer = new KeywordRecognizer(keywords.Keys.ToArray());
        keywordRecognizer.OnPhraseRecognized += KeywordRecognizer_OnPhraseRecognized;
        //开始识别
        keywordRecognizer.Start();
    }

    private void KeywordRecognizer_OnPhraseRecognized(PhraseRecognizedEventArgs args)
    {
        System.Action keywordAction;
        // 如果识别到关键词就调用
        if (keywords.TryGetValue(args.text, out keywordAction))
        {
            keywordAction.Invoke();
        }
    }

    void OnDestroy() 
    {
        keywordRecognizer.Dispose(); 
    }
}

下面是对 KeywordRecognizer.cs 脚本的一个应用：

Paste_Image.png

将KeywordRecognizer.cs 脚本中的关键词和动作操作的注册公开 Inspector 面板中，来代替在脚本中代码添加注册语音事件，这样可以在unity没运行的时候动态地添加语音命令。其脚本修改如下：

**KeywordRecognizer.cs**

using System.Collections.Generic;
using System.Linq;
using UnityEngine;
using UnityEngine.Events;
using UnityEngine.Windows.Speech;

namespace HoloToolkit
{
    public class KeywordManager : MonoBehaviour
    {
        //将此结构体暴露在Inspector面板中实现语音关键词和动作的添加
        public struct KeywordAndResponse
        {
            [Tooltip("The keyword to recognize.")]
            public string Keyword;
            [Tooltip("The UnityEvent to be invoked when the keyword is recognized.")]
            public UnityEvent Response;
        }

        // This enumeration gives the manager two different ways to handle the recognizer. Both will
        // set up the recognizer and add all keywords. The first causes the recognizer to start
        // immediately. The second allows the recognizer to be manually started at a later time.
       //这个枚举用于选择是立即进行语音识别 还是 过段时间进行手动设置进行语音识别
        public enum RecognizerStartBehavior { AutoStart, ManualStart };

        [Tooltip("An enumeration to set whether the recognizer should start on or off.")]
        public RecognizerStartBehavior RecognizerStart;
        [Tooltip("An array of string keywords and UnityEvents, to be set in the Inspector.")]
        public KeywordAndResponse[] KeywordsAndResponses;

        private KeywordRecognizer keywordRecognizer;
        private Dictionary<string, UnityEvent> responses;

        void Start()
        {
            if (KeywordsAndResponses.Length > 0)
            {
                // Convert the struct array into a dictionary, with the keywords and the keys and the methods as the values.
                // This helps easily link the keyword recognized to the UnityEvent to be invoked.
                //将struct数组转换为字典，将struct中的Keyword转换为字典的关键字，UnityEven转换为字典的方法
                responses = KeywordsAndResponses.ToDictionary(keywordAndResponse => keywordAndResponse.Keyword,
                                                              keywordAndResponse => keywordAndResponse.Response);

                //由注册的关键字来初始化KeywordRecognizer
                keywordRecognizer = new KeywordRecognizer(responses.Keys.ToArray());
                keywordRecognizer.OnPhraseRecognized += KeywordRecognizer_OnPhraseRecognized;
                
                //如果RecognizerStartBehavior枚举值是AutoStart的话，就立马开始语音识别
                if (RecognizerStart == RecognizerStartBehavior.AutoStart)
                {
                    keywordRecognizer.Start();
                }
            }
            else
            {
                Debug.LogError("Must have at least one keyword specified in the Inspector on " + gameObject.name + ".");
            }
        }

        void OnDestroy()
        {
            if (keywordRecognizer != null)
            {
                StopKeywordRecognizer();
                keywordRecognizer.OnPhraseRecognized -= KeywordRecognizer_OnPhraseRecognized;
                keywordRecognizer.Dispose();      //取消语音识别后必须Dispose掉，由GC回收，节省资源
            }
        }

        private void KeywordRecognizer_OnPhraseRecognized(PhraseRecognizedEventArgs args)
        {
            UnityEvent keywordResponse;

            // Check to make sure the recognized keyword exists in the methods dictionary, then invoke the corresponding method.
            // 如果识别到关键词就调用
            if (responses.TryGetValue(args.text, out keywordResponse))
            {
                keywordResponse.Invoke();
            }
        }

        /// <summary>
        /// Make sure the keyword recognizer is off, then start it.
        /// Otherwise, leave it alone because it's already in the desired state.
        /// </summary>
        public void StartKeywordRecognizer()
        {
            if (keywordRecognizer != null && !keywordRecognizer.IsRunning)
            {
                keywordRecognizer.Start();
            }
        }

        /// <summary>
        /// Make sure the keyword recognizer is on, then stop it.
        /// Otherwise, leave it alone because it's already in the desired state.
        /// </summary>
        public void StopKeywordRecognizer()
        {
            if (keywordRecognizer != null && keywordRecognizer.IsRunning)
            {
                keywordRecognizer.Stop();
            }
        }
    }
}

1. 在脚本中可以看到首先声明一个结构体，将其暴露在在Inspector面板，动态在unity中注册语音关键词和对应的动作函数，在 Start() 函数中将这个结构体相关的数组转换为一个Dictionary，将struct中的Keyword转换为字典的关键字，UnityEven转换为字典的方法。
2. 脚本声明了一个enum，给了两种不同的状态来处理识别，一个是立即开始语音识别，另一个是手动设置开始识别。
3. 当取消语音识别后，必须将 KeywordRecognizer 给 Dispose() 掉，随后GC会自动进行垃圾回收，以节省资源消耗。