美文网首页Android开发我爱编程
语音识别,语义理解一站式解决(android平台&ola

语音识别,语义理解一站式解决(android平台&ola

作者: ls0609 | 来源:发表于2017-08-01 18:59 被阅读248次

    转载请注明CSDN博文地址:http://blog.csdn.net/ls0609/article/details/71519203

    语音记账demo:http://blog.csdn.net/ls0609/article/details/72765789

    olami sdk语音识别语义理解做在线听书

    olamisdk实现了把录音或者文字转化为用户可以理解的json字符串,本文使用olami sdk做了一个在线听书的demo,用的是喜马拉雅的在线听书sdk.基于eclipse开发环境,libs目录下jar和so文件如下:

    olami-android-sdk.jar

    //olami sdk的jar

    afinal_0.5.1_bin.jar

    litepal.jar

    gson-2.2.4.jar

    okhttp-2.4.0.jar

    okhttp-urlconnection-2.2.0.jar

    okio-1.4.0.jar

    opensdk.jar//上面这几个都是喜马拉雅需要的jar

    libspeex.so//olami sdk需要用到speex压缩功能

    libxmediaplayer.so//喜马拉雅so

    libxmediaplayer_x.so

    //喜马拉雅so

    概述

    VoiceSdkService中定义了OlamiVoiceRecognizer语音识别引擎,通过点击MusicActivity的开始button启动录音,录音结果在VoiceSdkService中的onResult()回调中拿到识别的Json字符串,在processServiceMessage()函数中处理后找到要听书的名称,然后进入BookUtil进行搜索,搜索到结果后通知VoiceSdkService进行播放,并通知MusicActivity更新播放进度等信息。

    1.AndroidManifest.xml配置

    package="com.olami.musicdemo"

    android:versionCode="1"

    android:versionName="1.0" >

    android:minSdkVersion="8"

    android:targetSdkVersion="21" />

    android:name="com.olami.musicdemo.OlamiApplication"

    android:allowBackup="true"

    android:icon="@drawable/ic_launcher"

    android:label="@string/app_name"

    android:theme="@style/AppTheme" >

    android:name="app_key"

    android:value="b617866c20482d133d5de66fceb37da3"/>

    android:name="pack_id"

    android:value="com.app.test.android" />

    android:name=".MusicActivity"

    android:label="@string/app_name" >

    android:name=".VoiceSdkService"

    android:exported="true" >

    android:name=

    "com.ximalaya.ting.android.opensdk.player.service.XmPlayerService"

    />

    2.layout布局文件

    layout_musicview.xml

    TextView有两个,tv_name显示听书的名称, tv_totoal_time显示听书的总时间。

    ProgressBar实时刷新显示听书的进度

    xmlns:tools="http://schemas.android.com/tools"

    android:layout_width="fill_parent"

    android:layout_height="wrap_content"

    android:background="@android:color/transparent">

    android:text="name"

    android:layout_width="wrap_content"

    android:layout_height="wrap_content"

    android:layout_marginTop="5dp"

    android:layout_centerHorizontal="true"

    android:id="@+id/tv_name"/>

    style="?android:attr/progressBarStyleHorizontal"

    android:layout_width="fill_parent"

    android:layout_height="wrap_content"

    android:layout_below="@+id/tv_name"

    android:layout_marginTop="10dp"

    android:layout_marginLeft="20dp"

    android:layout_marginRight="20dp"

    android:id="@+id/progressbar_music"/>

    android:text="total_time"

    android:layout_width="wrap_content"

    android:layout_height="wrap_content"

    android:layout_below="@+id/progressbar_music"

    android:layout_marginTop="10dp"

    android:layout_centerHorizontal="true"

    android:id="@+id/tv_total_time"/>

    activity_music.xml

    xmlns:tools="http://schemas.android.com/tools"

    android:layout_width="match_parent"

    android:layout_height="match_parent"

    android:paddingLeft="@dimen/activity_horizontal_margin"

    android:paddingRight="@dimen/activity_horizontal_margin"

    tools:context="com.olami.musicdemo.MusicActivity" >

    android:id="@+id/tv_inputText"

    android:layout_width="wrap_content"

    android:layout_height="wrap_content"

    android:layout_marginTop="5dp"

    android:text="输入:" />

    android:id="@+id/tv_volume"

    android:layout_width="wrap_content"

    android:layout_height="wrap_content"

    android:layout_alignLeft="@+id/tv_inputText"

    android:layout_below="@+id/tv_inputText"

    android:layout_marginTop="40dp"

    android:text="音量:" />

    android:id="@+id/tv_result"

    android:layout_width="wrap_content"

    android:layout_height="wrap_content"

    android:layout_below="@+id/tv_volume"

    android:layout_marginTop="20dp"

    android:maxLines="15"

    android:ellipsize="end"

    android:text="服务器返回sentence:"

    android:visibility="visible" />

    android:id="@+id/music_view"

    android:layout_width="fill_parent"

    android:layout_height="80dp"

    android:layout_centerInParent="true"

    >

    android:id="@+id/et_content"

    android:layout_width="wrap_content"

    android:layout_height="40dp"

    android:layout_above="@+id/btn_stop"

    android:layout_alignLeft="@+id/tv_inputText"

    android:layout_marginBottom="10dp"

    android:layout_toLeftOf="@+id/btn_send"

    android:background="#E7E7E7"

    android:singleLine="true"

    android:text="上海的天气" />

    android:id="@+id/btn_send"

    android:layout_width="wrap_content"

    android:layout_height="wrap_content"

    android:layout_alignBaseline="@+id/et_content"

    android:layout_alignBottom="@+id/et_content"

    android:layout_alignParentRight="true"

    android:text="提交" />

    android:id="@+id/btn_start"

    android:layout_width="wrap_content"

    android:layout_height="wrap_content"

    android:layout_alignParentBottom="true"

    android:layout_centerHorizontal="true"

    android:text="开始" />

    android:id="@+id/btn_stop"

    android:layout_width="wrap_content"

    android:layout_height="wrap_content"

    android:layout_alignLeft="@+id/et_content"

    android:layout_alignParentBottom="true"

    android:text="停止" />

    android:id="@+id/btn_cancel"

    android:layout_width="wrap_content"

    android:layout_height="wrap_content"

    android:layout_alignParentBottom="true"

    android:layout_toRightOf="@+id/et_content"

    android:text="取消" />

    自定义MusicView比较简单,代码如下:

    public class MusicView extendsRelativeLayout{

    private Context mContext;

    private Handler mHandler;

    private TextView mTextViewName;

    private TextView mTextViewTotalTime;

    private ProgressBar mProgressBar;

    public MusicView(Context context,AttributeSet attrs) {

    super(context,attrs);

    LayoutInflater inflater =(LayoutInflater) context.getSystemService(

    context.LAYOUT_INFLATER_SERVICE);

    RelativeLayout view = (RelativeLayout) inflater.inflate

    (R.layout.layout_musicview,this,true);

    mTextViewName = (TextView) view.findViewById(R.id.tv_name);

    mTextViewTotalTime = (TextView) view.findViewById(R.id.tv_total_time);

    mProgressBar = (ProgressBar)view.findViewById(R.id.progressbar_music);

    }

    public void initMusicView(Context context,Handler handler)

    {

    mContext = context;

    mHandler = handler;

    }

    public void setMusicName(String name)

    {//设置播放名称

    mTextViewName.setText(name);

    }

    public void setProgress(int progress)

    {//设置播放进度

    mProgressBar.setProgress(progress);

    }

    public void setTotalTime(String time)

    {//设置播放总时间

    mTextViewTotalTime.setText(time);

    }

    }

    布局效果图如下:

    3.MusicActivity和VoiceSdkService通信

    本文没有用bind

    service的方式实现activity和service的消息通信。

    MusicAcitity和VoiceSdkService中分别实现了一个CommunicationAssist的接口

    public interface CommunicationAssist {

    public void callBack(int what, int arg1, int arg2,Bundle data, Objectobj);

    }

    然后把他们分别实现CommunicationAssist接口的变量注册到OlamiApplication,这样通过OlamiApplication实现了MusicAcitity和VoiceSdkService桥接。

    3.1OlamiApplication

    1)注册MusicActivityVoiceSdkService的回调

    public voidsetActivityToServiceListener(CommunicationAssist listener)

    {

    ActivityToServiceListener = listener;

    }

    这个是在VoiceSdkService中调用setActivityToServiceListener(),把VoiceSdkService中的VoiceSdkComAssist注册到application中,MusicActivity中可以通过getActivityToServiceListener

    这个函数回调向VoiceSdkService发送消息。

    2)注册VoiceSdkServiceMusicActivity回调

    public voidsetServiceToActivityListener(CommunicationAssist listener)

    {

    mServiceToActivityListener = listener;

    }

    这个是在MusicAcitivity中调用setServiceToActivityListener(),这样在VoiceSdkService中就可以通过getServiceToActivityListener()获得回调向MusciActivity发送消息。

    3.2

    MusicActivity

    public class MusicActivity extends Activity{

    private Handler mHandler;

    private Handler mInComingHandler;

    private ActivityComAssist mActivityComAssist;

    private Button mBtnStart;

    private Button mBtnStop;

    private Button mBtnCancel;

    private Button mBtnSend;

    private EditText mEditText;

    private TextView mTextView;

    private TextView mInputTextView;

    private TextView mTextViewVolume;

    private BookUtil mBookUtil = null;

    private MusicView mMusicView = null;

    @Override

    protected void onCreate(Bundle savedInstanceState) {

    super.onCreate(savedInstanceState);

    setContentView(R.layout.activity_music);

    initHandler();//初始化handler用于内部消息处理

    initInComingHandler();//用于处理来自VoiceSdkService的消息

    initCommunicationAssist();//向application注册消息回调,VoiceSdkSerive可以

    //通过getServiceToActivityListener()获得回调向MusicActivity发送消息

    initView();//初始化view控件

    Intent intent = new Intent();

    intent.setClass(MusicActivity.this, VoiceSdkService.class);

    startService(intent);//启动后台服务

    }

    private void initView()

    {

    mBtnStart = (Button) findViewById(R.id.btn_start);

    mBtnStop = (Button) findViewById(R.id.btn_stop);

    mBtnCancel = (Button) findViewById(R.id.btn_cancel);

    mBtnSend = (Button) findViewById(R.id.btn_send);

    mInputTextView = (TextView) findViewById(R.id.tv_inputText);

    mEditText = (EditText) findViewById(R.id.et_content);

    mTextView = (TextView) findViewById(R.id.tv_result);

    mTextViewVolume = (TextView) findViewById(R.id.tv_volume);

    mBtnStart.setOnClickListener(newOnClickListener(){

    @Override

    public void onClick(View v) {

    sendMessageToService(MessageConst.CLIENT_ACTION_START_RECORED,0,0,null,null);

    }

    });

    mBtnStop.setOnClickListener(new OnClickListener(){

    @Override

    public void onClick(View v) {

    sendMessageToService(MessageConst.CLIENT_ACTION_STOP_RECORED,0,0,null,null);

    mBtnStart.setText("开始");

    Log.i("led","MusicActivity mBtnStop onclick开始");

    }

    });

    mBtnCancel.setOnClickListener(new OnClickListener(){

    @Override

    public void onClick(View v) {

    sendMessageToService(MessageConst.CLIENT_ACTION_CANCEL_RECORED,0,0,null,null);

    }

    });

    mBtnSend.setOnClickListener(new OnClickListener(){

    @Override

    public void onClick(View v) {

    sendMessageToService(

    MessageConst.CLIENT_ACTION_SENT_TEXT,0,0,null,mEditText.getText());

    mInputTextView.setText("文字:"+mEditText.getText());

    }

    });

    mMusicView = (MusicView) findViewById(R.id.music_view);

    //if(mMusicView != null)

    //mMusicView.initMusicView(MusicActivity.this,mHandler);

    }

    private void initHandler()

    {

    mHandler = new Handler(){

    @Override

    public voidhandleMessage(Message msg)

    {

    switch (msg.what){

    caseMessageConst.CLIENT_ACTION_START_RECORED:

    break;

    default:

    break;

    }

    }

    };

    }

    //InComingHandler收到来自VoiceSdkService的消息用于更新界面,

    //包括开始录音,结束录音,播放的书的名称和进度,音量等信息。

    private void initInComingHandler()

    {

    mInComingHandler = new Handler(){

    @Override

    public void handleMessage(Message msg)

    {

    switch (msg.what){

    caseMessageConst.CLIENT_ACTION_START_RECORED:

    mBtnStart.setText("录音中");

    Log.i("led","MusicActivity录音中");

    break;

    caseMessageConst.CLIENT_ACTION_STOP_RECORED:

    mBtnStart.setText("识别中");

    Log.i("led","MusicActivity识别中");

    break;

    caseMessageConst.CLIENT_ACTION_CANCEL_RECORED:

    mBtnStart.setText("开始");

    mTextView.setText("已取消");

    break;

    caseMessageConst.CLIENT_ACTION_ON_ERROR:

    mTextView.setText("错误代码:"+msg.arg1);

    mBtnStart.setText("开始");

    break;

    caseMessageConst.CLIENT_ACTION_UPDATA_VOLUME:

    mTextViewVolume.setText("音量: "+msg.arg1);

    break;

    caseMessageConst.SERVER_ACTION_RETURN_RESULT:

    //mTextView.setText(msg.obj.toString());

    mBtnStart.setText("开始");

    break;

    caseMessageConst.CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH:

    mBtnStart.setText("开始");

    mBookUtil =BookUtil.getInstance();

    mBookUtil.play(msg.arg1);

    break;

    caseMessageConst.CLIENT_ACTION_UPDATA_PLAYING_BOOK_NAME:

    mMusicView.setMusicName(msg.obj.toString());

    break;

    caseMessageConst.CLIENT_ACTION_UPDATE_BOOK_PROGRESS:

    int current = msg.arg1;

    int duration = msg.arg2;

    mMusicView.setProgress(current*100/duration);

    float time =duration/1000/60;

    mMusicView.setTotalTime("总时间:"+time);

    break;

    caseMessageConst.CLIENT_ACTION_UPDATA_INPUT_TEXT:

    if(msg.obj != null)

    mInputTextView.setText("文字:"+msg.obj.toString());

    break;

    caseMessageConst.CLIENT_ACTION_UPDATA_SERVER_MESSAGE:

    if(msg.obj != null)

    mTextView.setText("服务器返回sentence:"+msg.obj.toString());

    break;

    default:

    break;

    }

    }

    };

    }

    private void initCommunicationAssist()

    {//向Application注册VoiceSdkService到MusicActivity的回调

    mActivityComAssist = new ActivityComAssist();

    OlamiApplication.getInstance().setServiceToActivityListener(mActivityComAssist);

    }

    private void sendMessageToService(int what, int arg1, int arg2, Bundledata, Object obj)

    {//向VoiceSdkService发送消息

    if(OlamiApplication.getInstance().getActivityToServiceListener() !=null)

    OlamiApplication.getInstance().getActivityToServiceListener().callBack

    (what, arg1, arg2, data, obj);

    }

    private class ActivityComAssist implements CommunicationAssist{

    //实现CommunicationAssist借口,用于回调VoiceSdkService发送过来的消息

    @Override

    public void callBack(int what, int arg1, int arg2, Bundle data,Objectobj) {

    Message msg = Message.obtain(null, what);

    msg.arg1 = arg1;

    msg.arg2 = arg2;

    if (data != null)

    msg.setData(data);

    if (obj != null)

    msg.obj = obj;

    mInComingHandler.sendMessage(msg);

    }

    }

    @Override

    public void onDestroy() {

    //退出应用,停止VoiceSdkService,会进行资源的释放

    super.onDestroy();

    Intent intent = new Intent();

    intent.setClass(MusicActivity.this, VoiceSdkService.class);

    stopService(intent);

    }

    }

    3.3

    VoiceSdkService

    @Override

    public void onCreate() {

    initHandler();//用于内部消息处理

    initInComingHandler();//用于处理来自MusicActivity的消息

    initCommunicationAssist();//向application注册消息回调,这样MusicActivity可

    //以通过getActivityToServiceListener()回调向VoiceSdkService发送消息

    initViaVoiceRecognizerListener();//初始化录音识别回调listener

    init();//olami录音识别引擎初始化

    initXmly();//喜马拉雅初始化

    }

    public void init()

    {

    initHandler();

    mOlamiVoiceRecognizer = new OlamiVoiceRecognizer(VoiceSdkService.this);

    TelephonyManager telephonyManager=(TelephonyManager)this.getSystemService(

    (this.getBaseContext().TELEPHONY_SERVICE);

    String imei=telephonyManager.getDeviceId();

    mOlamiVoiceRecognizer.init(imei);//设置身份标识,可以填null

    mOlamiVoiceRecognizer.setListener(mOlamiVoiceRecognizerListener);//设置识别结果回调listener

    mOlamiVoiceRecognizer.setLocalization(

    OlamiVoiceRecognizer.LANGUAGE_SIMPLIFIED_CHINESE);//设置支持的语音类型,优先选择中文简体

    mOlamiVoiceRecognizer.setAuthorization("51a4bb56ba954655a4fc834bfdc46af1",

    "asr","68bff251789b426896e70e888f919a6d","nli");

    //注册Appkey,在olami官网注册应用后生成的appkey

    //注册api,请直接填写“asr”,标识语音识别类型

    //注册secret,在olami官网注册应用后生成的secret

    //注册seq,请填写“nli”

    mOlamiVoiceRecognizer.setVADTailTimeout(2000);//录音时尾音结束时间,建议填//2000ms

    //设置经纬度信息,不愿上传位置信息,可以填0

    mOlamiVoiceRecognizer.setLatitudeAndLongitude(31.155364678184498,121.34882432933009);

    }

    定义OlamiVoiceRecognizerListener

    onError(int errCode)//出错回调,可以对比官方文档错误码看是什么错误

    onEndOfSpeech()//录音结束

    onBeginningOfSpeech()//录音开始

    onResult(String result, int type)//result是识别结果JSON字符串

    onCancel()//取消识别,不会再返回识别结果

    onUpdateVolume(int volume)//录音时的音量,1-12个级别大小音量

    下面是VoiceSdkService完整代码:

    public class VoiceSdkService extends Service{

    private Handler mHandler;

    private Handler mInComingHandler;

    private VoiceSdkComAssist mVoiceSdkComAssist;

    private OlamiVoiceRecognizer mOlamiVoiceRecognizer;

    private OlamiVoiceRecognizerListener mOlamiVoiceRecognizerListener;

    privateBookUtil mBookUtil = null;

    private boolean mIsRecordPause = false;

    @Override

    public void onCreate() {

    initHandler();

    initInComingHandler();

    initCommunicationAssist();

    initViaVoiceRecognizerListener();

    init();

    initXmly();

    }

    @Override

    public IBinder onBind(Intent intent) {

    // TODO Auto-generated method stub

    return null;

    }

    public void init()

    {

    initHandler();

    mOlamiVoiceRecognizer = new OlamiVoiceRecognizer(VoiceSdkService.this);

    TelephonyManager telephonyManager=(TelephonyManager)this.getSystemService(

    this.getBaseContext().TELEPHONY_SERVICE);

    String imei=telephonyManager.getDeviceId();

    mOlamiVoiceRecognizer.init(imei);//set null if you do not want to notifyolami server.

    mOlamiVoiceRecognizer.setListener(mOlamiVoiceRecognizerListener);

    mOlamiVoiceRecognizer.setLocalization(

    OlamiVoiceRecognizer.LANGUAGE_SIMPLIFIED_CHINESE);

    mOlamiVoiceRecognizer.setAuthorization(

    "51a4bb56ba954655a4fc834bfdc46af1",

    "asr","68bff251789b426896e70e888f919a6d","nli");

    mOlamiVoiceRecognizer.setVADTailTimeout(2000);

    mOlamiVoiceRecognizer.setLatitudeAndLongitude(31.155364678184498,121.34882432933009);

    }

    private void initHandler()

    {

    mHandler = new Handler(){

    @Override

    public void handleMessage(Message msg)

    {

    switch (msg.what){

    caseMessageConst.CLIENT_ACTION_START_RECORED:

    sendMessageToActivity(MessageConst.CLIENT_ACTION_START_RECORED,0,0,null,null);

    break;

    case MessageConst.CLIENT_ACTION_STOP_RECORED:

    sendMessageToActivity(MessageConst.CLIENT_ACTION_STOP_RECORED,0,0,null,null);

    break;

    caseMessageConst.CLIENT_ACTION_ON_ERROR:

    sendMessageToActivity(MessageConst.CLIENT_ACTION_ON_ERROR,msg.arg1,0,null,null);

    break;

    caseMessageConst.CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH:

    sendMessageToActivity(MessageConst.

    CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH,msg.arg1, 0, null, msg.obj);

    break;

    caseMessageConst.CLIENT_ACTION_UPDATA_PLAYING_BOOK_NAME:

    sendMessageToActivity(MessageConst.

    CLIENT_ACTION_UPDATA_PLAYING_BOOK_NAME, msg.arg1, 0, null, msg.obj);

    break;

    caseMessageConst.CLIENT_ACTION_UPDATE_BOOK_PROGRESS:

    sendMessageToActivity(MessageConst.

    CLIENT_ACTION_UPDATE_BOOK_PROGRESS,msg.arg1, msg.arg2, null, null);

    break;

    caseMessageConst.CLIENT_ACTION_CANCEL_RECORED:

    sendMessageToActivity(MessageConst.

    CLIENT_ACTION_CANCEL_RECORED,msg.arg1, msg.arg2, null, null);

    break;

    default:

    break;

    }

    }

    };

    }

    private void initInComingHandler()

    {

    mInComingHandler = new Handler(){

    @Override

    public void handleMessage(Message msg)

    {

    switch (msg.what){

    caseMessageConst.CLIENT_ACTION_START_RECORED:

    if(mOlamiVoiceRecognizer !=null)

    mOlamiVoiceRecognizer.start();

    break;

    caseMessageConst.CLIENT_ACTION_STOP_RECORED:

    if(mOlamiVoiceRecognizer !=null)

    mOlamiVoiceRecognizer.stop();

    break;

    caseMessageConst.CLIENT_ACTION_CANCEL_RECORED:

    if(mOlamiVoiceRecognizer !=null)

    mOlamiVoiceRecognizer.cancel();

    break;

    case MessageConst.CLIENT_ACTION_SENT_TEXT:

    if(mOlamiVoiceRecognizer !=null)

    mOlamiVoiceRecognizer.sendText(msg.obj.toString());

    break;

    }

    }

    };

    }

    private void initViaVoiceRecognizerListener()

    {

    mOlamiVoiceRecognizerListener = new OlamiVoiceRecognizerListener();

    }

    private class OlamiVoiceRecognizerListener implementsIOlamiVoiceRecognizerListener{

    @Override

    public void onError(int errCode) {

    mHandler.sendMessage(mHandler.obtainMessage(

    MessageConst.CLIENT_ACTION_ON_ERROR,errCode,0));

    }

    @Override

    public void onEndOfSpeech() {

    mHandler.sendEmptyMessage(MessageConst.CLIENT_ACTION_STOP_RECORED);

    if(mIsRecordPause)

    {

    mIsRecordPause = false;

    mBookUtil.resumePlay();

    }

    }

    @Override

    public void onBeginningOfSpeech() {

    if(mBookUtil.isPlaying())

    {

    mBookUtil.pause();

    mIsRecordPause = true;

    }

    mHandler.sendEmptyMessage(MessageConst.CLIENT_ACTION_START_RECORED);

    }

    @Override

    public void onResult(String result, int type) {

    sendMessageToActivity(MessageConst.SERVER_ACTION_RETURN_RESULT,type,0,null,result);

    processServiceMessage(result);

    }

    @Override

    public void onCancel() {

    mHandler.sendEmptyMessage(MessageConst.CLIENT_ACTION_CANCEL_RECORED);

    }

    @Override

    public void onUpdateVolume(int volume) {

    sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_VOLUME,volume,0,null,null);

    }

    }

    private void initCommunicationAssist()

    {

    mVoiceSdkComAssist = new VoiceSdkComAssist();

    OlamiApplication.getInstance().setActivityToServiceListener(mVoiceSdkComAssist);

    }

    private void initXmly()

    {

    if(mBookUtil == null)

    {

    mBookUtil = BookUtil.getInstance();

    mBookUtil.init(VoiceSdkService.this);

    mBookUtil.setHandler(mHandler);

    }

    }

    private void processServiceMessage(String message)

    {

    String input = null;

    String serverMessage = null;

    try{

    JSONObject jsonObject = new JSONObject(message);

    JSONArray jArrayNli = jsonObject.optJSONObject("data").optJSONArray("nli");

    JSONObject jObj = jArrayNli.optJSONObject(0);

    JSONArray jArraySemantic = null;

    if(message.contains("semantic"))

    jArraySemantic =jObj.getJSONArray("semantic");

    else{

    input =jsonObject.optJSONObject("data").optJSONObject("asr").

    optString("result");

    sendMessageToActivity(MessageConst.

    CLIENT_ACTION_UPDATA_INPUT_TEXT, 0, 0, null, input);

    serverMessage =jObj.optJSONObject("desc_obj").opt("result").toString();

    sendMessageToActivity(MessageConst.

    CLIENT_ACTION_UPDATA_SERVER_MESSAGE, 0, 0, null, serverMessage);

    return;

    }

    JSONObject jObjSemantic;

    JSONArray jArraySlots;

    JSONArray jArrayModifier;

    String type = null;

    String songName = null;

    String singer = null;

    if(jObj != null) {

    type =jObj.optString("type");

    if("musiccontrol".equals(type))

    {

    jObjSemantic =jArraySemantic.optJSONObject(0);

    input =jObjSemantic.optString("input");

    jArraySlots =jObjSemantic.optJSONArray("slots");

    jArrayModifier =jObjSemantic.optJSONArray("modifier");

    String modifier =(String)jArrayModifier.opt(0);

    if((jArrayModifier != null)&& ("play".equals(modifier)))

    {

    if(jArraySlots != null)

    for(inti=0,k=jArraySlots.length(); i

    {

    JSONObject obj =jArraySlots.getJSONObject(i);

    String name =obj.optString("name");

    if("singer".equals(name))

    singer =obj.optString("value");

    elseif("songname".equals(name))

    songName =obj.optString("value");

    }

    }else if((modifier != null)&& ("stop".equals(modifier)))

    {

    if(mBookUtil != null)

    if(mBookUtil.isPlaying())

    mBookUtil.stop();

    }else if((modifier != null)&& ("pause".equals(modifier)))

    {

    if(mBookUtil != null)

    if(mBookUtil.isPlaying())

    mBookUtil.pause();

    }else if((modifier != null)&& ("resume_play".equals(modifier)))

    {

    if(mBookUtil != null)

    mBookUtil.resumePlay();

    }else if((modifier != null)&& ("add_volume".equals(modifier)))

    {

    if(mBookUtil != null)

    mBookUtil.addVolume();

    }else if((modifier != null)&& ("del_volume".equals(modifier)))

    {

    if(mBookUtil != null)

    mBookUtil.delVolume();

    }else if((modifier != null)&& ("next".equals(modifier)))

    {

    if(mBookUtil != null)

    mBookUtil.next();

    }else if((modifier != null)&& ("previous".equals(modifier)))

    {

    if(mBookUtil !=null)

    mBookUtil.prev();

    }else if((modifier != null)&& ("play_index".equals(modifier)))

    {

    int position = 0;

    if(jArraySlots != null)

    for(inti=0,k=jArraySlots.length(); i

    {

    JSONObjectobj = jArraySlots.getJSONObject(i);

    JSONObjectjNumDetial = obj.getJSONObject("num_detail");

    String index= jNumDetial.optString("recommend_value");

    position =Integer.parseInt(index) - 1;

    }

    if(mBookUtil != null)

    mBookUtil.skipTo(position);

    }

    }

    }

    if(songName != null)

    {

    if(singer != null)

    {

    }else{

    mBookUtil.searchBookAndPlay(songName,0,0);

    }

    }else if(singer != null)

    {

    mBookUtil.searchBookAndPlay(songName,0,0);

    }

    serverMessage =jObj.optJSONObject("desc_obj").opt("result").toString();

    }

    catch (Exception e)

    {

    e.printStackTrace();

    }

    //发送消息更新语音识别的文字

    sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_INPUT_TEXT, 0,0, null, input);

    //发送消息更新服务器返回的结果字符串

    sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_SERVER_MESSAGE,

    0, 0, null, serverMessage);

    }

    private void sendMessageToActivity(int what, int arg1, int arg2, Bundledata, Object obj)

    {

    if(OlamiApplication.getInstance().getServiceToActivityListener() !=null)

    OlamiApplication.getInstance().getServiceToActivityListener().

    callBack(what, arg1, arg2, data, obj);

    }

    private class VoiceSdkComAssist implements CommunicationAssist{

    @Override

    public void callBack(int what, int arg1, int arg2, Bundle data,Objectobj) {

    Message msg = Message.obtain(null, what);

    msg.arg1 = arg1;

    msg.arg2 = arg2;

    if (data != null)

    msg.setData(data);

    if (obj != null)

    msg.obj = obj;

    mInComingHandler.sendMessage(msg);

    }

    }

    @Override

    public void onDestroy(){

    super.onDestroy();

    if(mOlamiVoiceRecognizer != null)

    mOlamiVoiceRecognizer.destroy();

    if(mBookUtil != null)

    {

    mBookUtil.destroy();

    }

    }

    }

    3.4

    VoiceSdkServiceonResult的回调处理

    在VoiceSdkService.java中processServiceMessage(String message)用于处理onResult的回调数据。例如“我要听三国演义”返回如下数据:

    {

    "data": {

    "asr": {

    "result":"我要听三国演义",

    "speech_status":0,

    "final":true,

    "status":0

    },

    "nli": [

    {

    "desc_obj": {

    "result":"正在努力搜索中,请稍等",

    "status":0

    },

    "semantic": [

    {

    "app":"musiccontrol",

    "input":"我要听三国演义",

    "slots": [

    {

    "name":"songname",

    "value":"三国演义"

    }

    ],

    "modifier": [

    "play"

    ],

    "customer":"58df512384ae11f0bb7b487e"

    }

    ],

    "type":"musiccontrol"

    }

    ]

    },

    "status":"ok"

    }

    1)解析出nli中type类型是musiccontrol,这是语法返回app的类型,而这个在线听书的demo只关心musiccontrol这个app类型,其他的忽略。

    2)用户说的话转成文字是在asr中的result中获取

    3)在nli中的semantic中,input值是用户说的话,同asr中的result。

    modifier代表返回的行为动作,此处可以看到是play就是要求播放,slots中的数据表示歌曲名称是三国演义。

    那么动作是play,内容是歌曲名称是三国演义,在这个demo中调用

    mBookUtil.searchBookAndPlay(songName,0,0);会先查询,查询到结果会再发播放消息要求播放,我要听三国演义这个流程就走完了。

    4.BookUtil

    说一下搜索听书的实现过程

    public void searchBookInfo(StringbookName,final int index,final boolean isNeedPlay)

    {

    mBookName = bookName;

    Map param = new HashMap();

    param.put(DTransferConstants.SEARCH_KEY, bookName);

    param.put(DTransferConstants.CATEGORY_ID, "" + 3);//此处3代表搜索的是听书

    //param.put(DTransferConstants.PAGE, "" + mPageId);

    param.put(DTransferConstants.SORT, "asc");//返回列表的排序是正序还是逆序

    param.put(DTransferConstants.PAGE_SIZE, "" + PAGE_SIZE);//每页能返回多少个查询结果

    mPage = (index/PAGE_SIZE)+1;//当前在第几页

    mPlayerManager = XmPlayerManager.getInstance(mContext);//喜马拉雅初始化部分

    mPlayerManager.init(mNotificationId, null);

    mPlayerManager.addPlayerStatusListener(mPlayerStatusListener);

    mPlayerManager.addAdsStatusListener(mAdsListener);

    CommonRequest.getSearchedAlbums(param, newIDataCallBack()

    {

    @Override

    public void onSuccess(SearchAlbumList object)

    {

    if (object != null && object.getAlbums() != null

    &&object.getAlbums().size() != 0)

    {

    if (mSearchAlbumList == null)

    {

    mSearchAlbumList = object;

    }

    else

    {

    mSearchAlbumList.getAlbums().addAll(object.getAlbums());

    }

    //mTrackAdapter.notifyDataSetChanged();

    Map map =new HashMap();

    map.put(

    DTransferConstants.ALBUM_ID,""+object.getAlbums().get(0).getId());

    map.put(DTransferConstants.SORT, "asc");

    map.put(DTransferConstants.PAGE, "" + mPage);

    map.put(DTransferConstants.PAGE_SIZE,"" + PAGE_SIZE);

    CommonRequest.getTracks(map,new IDataCallBack()

    {

    @Override

    public voidonSuccess(TrackList object)

    {

    mTrackList =object;

    mTotalCount =mTrackList.getTotalCount();

    if(mTrackList.getTracks().size() <= 0)

    return;

    String str = "专辑:"+mTrackList.getAlbumTitle()+

    get(0).getTrackTitle().toString();

    if(isNeedPlay)

    {

    mPosition =index % PAGE_SIZE;

    mHandler.sendMessage(mHandler.obtainMessageMessageConst.CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH,

    index %PAGE_SIZE,0));//此处mTrackList中已经查询出结果

    //向VoiceSdkService发送消息进行播放

    }

    else

    sendBookInfoToServer();

    }

    @Override

    public void onError(intcode, String message)

    {

    Log.i("ppp","error: "+message);

    sendBookInfoToServer();

    }

    });

    }

    }

    @Override

    public void onError(int code, String message)

    {

    Log.i("ppp","error: "+message);

    sendBookInfoToServer();

    }

    });

    }

    5.demo中支持的说法

    我想听西游记

    我要听西游记

    播放西游记

    听西游记

    我想听西游记这本书

    上一首

    上一回

    下一首

    下一回

    暂停/暂停播放

    继续/继续播放

    声音大一点

    声音小一点

    关闭/关闭播放

    用的是喜马拉雅测试账号,只支持听书的功能,查找歌曲的结果返回为空。

    6.源码下载链接

    https://github.com/ls0609/olami_musicdemo

    7.相关博客

    语音识别语义理解一站式解决(android平台&olami sdk)

    http://blog.csdn.net/ls0609/article/details/71519203

    olami开放平台语法编写简介:http://blog.csdn.net/ls0609/article/details/71624340

    olami开放平台语法官方介绍:https://cn.olami.ai/wiki/?mp=nli&content=nli2.html

    csdn博文:http://blog.csdn.net/ls0609/article/details/71519203

    相关文章

      网友评论

        本文标题:语音识别,语义理解一站式解决(android平台&ola

        本文链接:https://www.haomeiwen.com/subject/whcslxtx.html