美文网首页WebRTC
Android端WebRtc+Kurento详解

Android端WebRtc+Kurento详解

作者: BaeBae_675c | 来源:发表于2017-03-24 17:06 被阅读0次

    WebRtc是google开源的视频通话技术，Kurento是Kurento公司开源的媒体服务器。两者结合起来可以达到多人视频通话的效果。目前在git上Android端webrtc+Kurento的demo几乎没有，本文主要介绍一下如何将两者结合以及一些需要注意的地方。

    • 需要的库

    1. KurentoRoomAndroid: 官方地址为 https://github.com/nubomedia-vtt/kurento-room-client-android我们仅使用其中的除了libjinglepeerconnection的其他jar包。

    2. libjinglepeerconnection: 根据上面的kurentoRoom地址引用下来的库中是有libjinglepeerconnection的，但编译版本较早，新版的webrtc已经有些不同。自行编译webrtc难度较大，读者可以先使用这个版本:https://github.com/BaeBae33/webrtc_android(将相关类放到自己工程下可能会报错，把buildToolsVersion提升到25.0.0及以上即可)

    • 权限

        <uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" />
    <uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
    <uses-permission android:name="android.permission.RECORD_AUDIO" />
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />

    • 视频通道建立流程

    webrtc的流程网上很多，相信很多人都能大致看得懂。这里我就谈谈具体在android下如何实施。
    场景为用户A进入Kurento房间并接收B发起的视频流(B已经在房间，并且发布视频,假设我们就是用户A):

    1. 创建PeerConnectionFactory
      在多人视频通话中,我们只需要实例化一个factory。代码如下:
            //初始化PeerConnectionFactory，以后用于生产PeerConnection
        PeerConnectionFactory.initializeInternalTracer();
        PeerConnectionFactory.initializeFieldTrials("");
        if (!PeerConnectionFactory.initializeAndroidGlobals(
                activity.this, true, true, true)) {
            Log.e(TAG, "Failed to initializeAndroidGlobals");
        }
        options = new PeerConnectionFactory.Options();
        options.networkIgnoreMask = 0;
        factory = new PeerConnectionFactory(options);
        Log.d(TAG, "Peer connection factory created.");
        // Set default WebRTC tracing and INFO libjingle logging.
        // NOTE: this _must_ happen while |factory| is alive!
        Logging.enableTracing("logcat:", EnumSet.of(Logging.TraceLevel.TRACE_DEFAULT));
        Logging.enableLogToDebugOutput(Logging.Severity.LS_INFO);
    1. 连接websocket并加入房间，代码很简单，就不贴了。
    2. 加入房间成功后，在RoomListener.onRoomResponse内我们接收到类似这样的回调:
    {"value":[{"id":"1","streams":[{"id":"webcam"}]}],"sessionId":"il67bnrkjeduve2q1jd2klik8d"}

    字符串中存在"webcam"，说明id为1的用户正在房间内发布视频。我们可以从这里初始化相关信息并createOffer。我封装了一个 initpeer方法，大家可以参考一下(用户B的type为OTHER)。

    public void initPeer() {
        peer = new PeerConnectionClient();
        if (type.equals(Type.SELF)) {
            sparam = new PeerConnectionClient.SignalingParameters(getServers(), true, id, null, null);
            param = new PeerConnectionClient.
                    PeerConnectionParameters(true, false, true, 640, 480, 0, 0, "VP8",
                    false, false, 0, "opus", false, false, false, false, false, false, false, null);
            } else if (type.equals(Type.OTHER)) {
            sparam = new PeerConnectionClient.SignalingParameters(getServers(), false, id, null, null);
            param = new PeerConnectionClient.
                    PeerConnectionParameters(false, false, true, 640, 480, 0, 0, "VP8",
                    false, false, 0, "opus", false, false, false, false, false, false, false, null);
        }
        events = new PeerConnectionClient.PeerConnectionEvents() {
            @Override
            public void onLocalDescription(SessionDescription sdp) {
                // TODO Auto-generated method stub
                LogCat.i("onLocalDescription1:" + sdp.description);
                LogCat.i(type.toString());
                if (type.equals(Type.SELF)) {
                    roomApi.sendPublishVideo(sdp.description, false, 1);
                } else if (type.equals(Type.OTHER)) {
                    LogCat.i(type.toString());
                    roomApi.sendReceiveVideoFrom(id + "_webcam", sdp.description, 2);
                }
            }
            @Override
            public void onIceCandidate(IceCandidate candidate) {
                // TODO Auto-generated method stub
                LogCat.i("onIceCandidate:" + candidate.toString());
                LogCat.i("onIceCandidate:detail1:" + candidate.sdp + "," + candidate.sdpMid + "," + String.valueOf(candidate.sdpMLineIndex));
                roomApi.sendOnIceCandidate(id, candidate.sdp, candidate.sdpMid,
                        String.valueOf(candidate.sdpMLineIndex), 3);
            }

            @Override
            public void onIceCandidatesRemoved(IceCandidate[] candidates) {
                // TODO Auto-generated method stub
            }

            @Override
            public void onIceConnected() {
                // TODO Auto-generated method stub
            }

            @Override
            public void onIceDisconnected() {
                // TODO Auto-generated method stub
            }

            @Override
            public void onPeerConnectionClosed() {
                // TODO Auto-generated method stub
            }

            @Override
            public void onPeerConnectionStatsReady(StatsReport[] reports) {
                // TODO Auto-generated method stub
            }

         @Override
            public void onPeerConnectionError(String description) {
                // TODO Auto-generated method stub

            }
        };
        peer.createPeerConnectionFactory(factory, eglBase.getEglBaseContext(), param, events);
        if (type.equals(Type.SELF)) {
            VideoRenderer.Callbacks remoteRender = new VideoRenderer.Callbacks() {
                @Override
                public void renderFrame(VideoRenderer.I420Frame i420Frame) {
                    LogCat.i(i420Frame.toString());
                }
            };          peer.createPeerConnection(eglBase.getEglBaseContext(), proxyRenderer, remoteRender, sparam);
            peer.createOffer();
        } else if (type.equals(Type.OTHER)) {
            peer.createPeerConnection(eglBase.getEglBaseContext(), null, proxyRenderer, sparam);
            peer.createOffer();
        }
    }

    需要注意的是PeerConnectionEvents中一些回调如何处理，以及自己的peer与其他人的peer的一些参数区别。

    1. 在RoomListener.onRoomResponse回调中set sdpAnswer:
          SessionDescription sdpAnswer = new SessionDescription(SessionDescription.Type.ANSWER,(String) response.getObj().get("sdpAnswer"));
      peer.setRemoteDescription(sdpAnswer);
    1. 在RoomListener.onRoomNotification回调中addRemoteIceCandidate:
    if(notification.getMethod().equals(
            RoomListener.METHOD_ICE_CANDIDATE)){
        // TODO
        peer.addRemoteIceCandidate(iceCandidate);
    }

    如果代码没有错，到此视频通道应该就打通成功，并且能看到B的实时视频了。

    • 显示视频画面

    SurfaceViewRenderer与ProxyRenderer:
    SurfaceViewRenderer是显示的控件,ProxyRenderer是实现了VideoRenderer.Callbacks的一个类:

    private class ProxyRenderer implements VideoRenderer.Callbacks {
        private VideoRenderer.Callbacks target;

        synchronized public void renderFrame(VideoRenderer.I420Frame frame) {
            if (target == null) {
                Logging.d(TAG, "Dropping frame in proxy because target is null.");
                VideoRenderer.renderFrameDone(frame);
                return;
            }
            target.renderFrame(frame);
        }
        synchronized public void setTarget(VideoRenderer.Callbacks target) {
            this.target = target;
        }
    }

    我们可以通过setTarget方法将视频流显示到一个SurfaceViewRenderer上，也可以随时更换到另一个SurfaceViewRenderer上：

    proxyRenderer.setTarget(renderer);

    • 免费可用的STUN

    List<PeerConnection.IceServer> iceServers = new ArrayList<>();
iceServers.add(new PeerConnection.IceServer("stun:stun.xten.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.voipbuster.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.voxgratia.org:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.sipgate.net:10000"));
iceServers.add(new PeerConnection.IceServer("stun:stun.ekiga.net:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.ideasip.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.schlund.de:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.voiparound.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.voipbuster.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.voipstunt.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:numb.viagenie.ca:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.counterpath.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.1und1.de:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.gmx.net:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.bcs2005.net:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.callwithus.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.counterpath.net:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.internetcalls.com:3478"));
iceServers.add(new PeerConnection.IceServer("stun:stun.voip.aebc.com:3478"));

    • 可能遇到的问题

    1. 为什么IceConnectionState一直停在checking?
      打洞不通 :(
    2. 为什么只有声音没有画面?
      有声音说明打通了，但视频流没有放到控件上。确保这几点正确:
      • proxyRenderer设置到SurfaceViewRenderer上，并且传递到了PeerConncetionClient内部。
      • PCObserver.onAddStream中对stream做了处理:
    @Override
        public void onAddStream(final MediaStream stream) {
            Log.d(TAG, "onAddStream");
            executor.execute(new Runnable() {
                @Override
                public void run() {
                    if (peerConnection == null || isError) {
                        return;
                    }
                    if (stream.audioTracks.size() > 1 || stream.videoTracks.size() > 1) {
                        reportError("Weird-looking stream: " + stream);
                        return;
                    }
                    if (stream.videoTracks.size() == 1) {
                        Log.i(TAG, "onAddStream Success");
                        remoteVideoTrack = stream.videoTracks.get(0);
                        remoteVideoTrack.setEnabled(renderVideo);
                        remoteVideoTrack.addRenderer(new VideoRenderer(remoteRender));
                    }
                }
            });
        }

    - 如果是自己的画面出不来，看看是不是没有createVideoTrack:

    if (videoCallEnabled) {
            mediaStream = factory.createLocalMediaStream("ARDAMS");
            if (videoCapturer == null) {
                reportError("Failed to open camera");
                return;
            }
            mediaStream.addTrack(createVideoTrack(videoCapturer));
            mediaStream.addTrack(createAudioTrack());
            peerConnection.addStream(mediaStream);
            findVideoSender();
        }

    - 如果是其他人的画面出不来，看看sdpMediaConstraints添加OfferToReceiveVideo了没:

    if (videoCallEnabled) {//videoCallEnabled || peerConnectionParameters.loopback
            sdpMediaConstraints.mandatory.add(
                    new MediaConstraints.KeyValuePair("OfferToReceiveVideo", "false"));
            sdpMediaConstraints.mandatory.add(
                    new MediaConstraints.KeyValuePair("OfferToReceiveAudio", "false"));
        } else {
            sdpMediaConstraints.mandatory.add(
                    new MediaConstraints.KeyValuePair("OfferToReceiveVideo", "true"));
            sdpMediaConstraints.mandatory.add(
                    new MediaConstraints.KeyValuePair("OfferToReceiveAudio", "true"));
        }

    • 总结

    kurento android之路不易，但更多的坑在webrtc中。比如我遇到的nativeFreeFactory(nativeFactory)崩溃，根据日志原因在native层没有分离线程。看样子要改源码了。

    相关文章

      网友评论

        本文标题：Android端WebRtc+Kurento详解

        本文链接：https://www.haomeiwen.com/subject/jmrtottx.html

        延伸阅读

        深度阅读

        • 您也可以注册成为美文阅读网的作者，发表您的原创作品、分享您的心情！

        栏目导航

        热点阅读

        WebRTC

        关于我们|服务条款|联系我们|Android端WebRtc+Kurento详解|投稿指南|网站地图|RSS订阅|排版工具|手机版

        提供经典美文摘抄,优美散文欣赏,现代诗歌精选,短篇小说,心情随笔,表白情书范文,故事会在线阅读欣赏

        Copyright © 2014-2023 Haomeiwen.com All Rights Reserved. 好美文阅读网 版权所有

        备案信息：桂公网安备 45052102000051号 · 桂ICP备13007215号-3

        本站所收录作品、热点评论等信息部分来源互联网，目的只是为了系统归纳学习和传递资讯

        所有作品版权归原创作者所有，与本站立场无关，如不慎侵犯了你的权益，请联系我们告知，我们将做删除处理！