如何创建 WebRTC 会话二

作者: 老瓦在霸都 | 来源:发表于2021-03-21 20:31 被阅读0次

WebRTC 是一套基于 Web 的实时通信解决方案，通过浏览器内置的 API 来支持音视频通道的搭建。

简而言之，先在信令通道协商出彼此的媒体和通信参数, 再通过媒体通道来传输音视频媒体数据。这一套媒体会话的搭建流程定义为 “JavaScript Session Establishment Protocol” JavaScript 会话创建协议

WebRTC 的信令，媒体以及数据通道

首先看一下 WebRTC 的实体之间的拓扑结构

WebRTC 协议栈如下图所示, 基本上有三个通道需要建立

Signal 信令通道, 可以通过 TCP/TLS + HTTP 和 WebSocket 来传输信令消息
Media 媒体通道, 可以通过 UDP + DTLS/SRTP 来传输媒体
Data 数据通道, 可以通过 UDP + DTLS + SCTP 来传输数据

对于媒体传输层，WebRTC 规定了用 ICE/STUN/TURN 来连通，用 DTLS 来协商 SRTP 密钥，用 SRTP 来传输媒体数据, 用 SCTP 来传输应用数据。

而在信令层，WebRTC 并未指定，各个应用可以用自己喜欢的信令协议来进行媒体协商，一般都是用 SDP 来通过 HTTP, WebSocket 或 SIP 协议承载具体的媒体会话描述。

信令和媒体通道是必需的，数据通道和信令通道可以合并。

会话创建过程(JSEP) 与两个最重要的状态机

在客户端即浏览器需要维护两个状态机

信令状态机

通信的双方需要将 SDP 进行交换以了解彼此的媒体通信能力。本地 Local 先创建并设置 SDP Offer, 状态为 "have-local-offer" , 发送给远端(SetRemote),远端服务收到 Offer, 再根据自己的能力结合对方提供的能力,生成一个 SDP answer , 然后发回给对端.于是当对方收到 Answer 并设置给自己的 RTCPeerConnection, 这个状态就会变成 "stable"

signal states

具体的交互如下图所示, Alice 要与 Bob 通信就要先交换 SDP, 进行一轮 Offer/Answer 的协商

ICE 连接状态机

信令通道搭建好进行 SDP 媒体能力的协商, 还要进行媒体通道的搭建, 首要的就是要创建好连接,能不能连接需要检查. 也即要将对收集到的候选者对 Candidate-pair 进行检查 "checking",如果能连通, 最终状态转换为 "connected"

通过 ICE/STUN/TURN 协议，将位于防火墙，准确地来说是 NAT (网络地址转换器) 之后的通信双方连接起来

可以象下面这样在SDP 在携带地址信息(ice-candidate: ip+port)

让我们来举一个具体例子, 客户端初始界面如下:

1. 我们要写一个上面提到的信令服务器, 同时也作为一个网页服务器

nodejs 有强大的 express 和 socket.io 库,可以帮助我们轻松实现一个视频聊天服务器。
express 是非常流行的 nodejs Web 框架，它简单易用， socket.io 是一个支持浏览器和服务器之间的实时、双向和基于事件的通信的 JavaScript 库。它包括 nodejs 服务器和Javascript客户端库。

第一步创建 package.json

{
  "name": "webrtc_primer",
  "version": "0.1.0",
  "description": "video_chat_demo",
  "main": "video_chat_server.js",
  "scripts": {
    "test": "echo \"Error: no test specified! Configure in package.json\" && exit 1",
    "start": "nodemon video_chat_server.js"
  },
  "repository": {
    "type": "git",
    "url": "https://github.com/walterfan/webrtc_primer"
  },
  "keywords": [
    "WebRTC"
  ],
  "author": "Simon Pietro Romano",
  "license": "BSD",
  "dependencies": {
    "body-parser": "^1.18.3",
    "express": "^4.16.3",
    "log4js": "^6.3.0",
    "moment": "^2.29.1",
    "nc": "^1.0.3",
    "node-static": "~0.7.11",
    "nodemon": "^1.17.5",
    "socket.io": "~3.0.4",
    "sqlite3": "^5.0.2",
    "webrtc-adapter": "^7.7.0"
  },
  "bugs": {
    "url": "https://github.com/spromano/WebRTC_Book/issues"
  },
  "homepage": "https://github.com/spromano/WebRTC_Book"
}

第二步：创建 video_chat_server.js

这里我用 express 框架来启动一个web 网页服务器，注意这里需要使用 https 协议。从安全性考虑，WebRTC是不允许用 http 协议的

首先使用openssl 来生成一个私钥和自签名的证书

openssl req \
       -newkey rsa:2048 -nodes -keyout domain.key \
       -x509 -days 365 -out domain.crt

你也可以跳过这一步，使用我生成好的。
添加一个文件 video_chat_server.js

const fs = require('fs');
const http = require('http');
const https = require('https');
const bodyParser = require('body-parser');
//const sqlite3 = require('sqlite3');

const moment = require('moment');
const express = require('express');
const path = require('path');

const log4js = require("log4js");

log4js.configure({
  appenders: { 
        'stdout': { type: 'stdout' },  
        'video_chat': { type: "file", filename: "video_chat.log" } 
  },
  categories: { default: { 
      appenders: ["stdout","video_chat"], 
      level: "info" } 
  }
});
 
const logger = log4js.getLogger("video_chat");

const options = {
    index: "video_chat_demo.html"
  };
  
const httpsPort = 8183;

const certificate = fs.readFileSync('./domain.crt', 'utf8');
const privateKey  = fs.readFileSync('./domain.key', 'utf8');

const credentials = {key: privateKey, cert: certificate};

const app = express();

app.use('/', express.static(path.join(__dirname, '/'), options));

const httpsServer = https.createServer(credentials, app);

console.log(`video chart serve on https://localhost:${httpsPort}`);
httpsServer.listen(httpsPort);

第三步：引入 socket.io 库，添加对于客户端连接的管理

主要就两件事：
一是把有相同房间名称的参会者的连接加入(join) 到一个房间(room)
二是在参会者之间广播消息

var io = require('socket.io')(httpsServer);

function getParticipantsOfRoom(roomId, namespace) {
    
    var count = 0;
    var ns = io.of(namespace||"/");    // the default namespace is "/"
 
    for (let [key, value] of ns.adapter.rooms) {
        
        if(key === roomId) {
            count += value.size;
        } 
    }
    
    return count;
}

// Let's start managing connections...
io.sockets.on('connection', function (socket){

        // Handle 'message' messages
        socket.on('message', function (message) {
                log('Server --> got message: ', message);
                logger.info('will broadcast message:', message);
                // channel-only broadcast...
                //socket.broadcast.to(socket.channel).emit('message', message);
                socket.broadcast.emit('message', message);

        });

        // Handle 'create or join' messages
        socket.on('create or join', function (room) {

                var numClients =  getParticipantsOfRoom(room);

                log('Server --> Room ' + room + ' has ' + numClients + ' client(s)');
                log('Server --> Request to create or join room', room);

                // First client joining...
                if (numClients == 0){
                        socket.join(room);
                        socket.emit('created', room);
                        logger.info(room + " created: " + numClients);
                } else if (numClients == 1) {
                        // Second client joining...
                        io.sockets.in(room).emit('join', room);
                        socket.join(room);
                        socket.emit('joined', room);
                        logger.info(room + " joined: " + numClients);
                } else { // max two clients
                        socket.emit('full', room);
                        logger.info(room + " full: " + numClients);
                }
        });

        function log(){
            var array = ["* " + moment().format() + ">>> "];
            for (var i = 0; i < arguments.length; i++) {
                array.push(arguments[i]);
            }
            socket.emit('log', array);
        }
});

2. 写一个支持视频和文本聊天的网页

这个网页就是实现一开始我展示的那个界面

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<!-- 省略一些引入的 js 和 css 文件-->
</head>
<body>

<nav class="navbar navbar-default navbar-static-top">
</nav>

<div class="container">
    <div class="row">
        <div class="col-lg-12">
            <div class="page-header">
                <h1>WebRTC example of Video Chat  </h1>
                <p> Please run `node video_chart_server.js` firstly. they are test cases in local</p>
            </div>
            <div class="container" id="details">
          <div class="row">

              <div class="col-lg-12">

                  <div>
                      <label>Room Name: </label>
                      <input type="text" id="roomName"/>
                      <button class="btn btn-default" autocomplete="off" id="openButton">Join</button>
                      <button class="btn btn-default" autocomplete="off" id="closeButton">Leave</button>

                      <button class="btn btn-default" autocomplete="off" id="startButton">Start Video</button>
                      <button class="btn btn-default" autocomplete="off" id="stopButton">Stop Video</button>

                </div>
                <br/>
              </div>
              <div class="col-lg-12">
                       <div id='mainDiv'>
                          <table border="1" width="100%">
                            <tr>
                              <th>
                                Local video
                              </th>
                              <th>
                                Remote video
                              </th>
                            </tr>
                            <tr>
                              <td>
                                <video id="localVideo" autoplay></video>
                              </td>
                              <td>
                                <video id="remoteVideo" autoplay></video>
                              </td>
                            </tr>
                            <tr>
                              <td align="center">
                                <textarea rows="4" cols="60"id="dataChannelSend" disabledplaceholder="This will be enabled once the data channel is up..."></textarea>
                              </td>
                              <td align="center">
                                <textarea rows="4" cols="60"id="dataChannelReceive" disabled></textarea>
                              </td>
                            </tr>
                            <tr>
                              <td align="center">
                                <button id="sendButton" disabled>Send</button>
                              </td>
                              <td>
                              </td>
                            </tr>
                          </table>
                        </div>
              </div>

          </div>
                
         
<script type="text/javascript" src="/socket.io/socket.io.js"></script>
<script type="text/javascript" src="js/old_adapter.js"></script>
<script type="text/javascript" src="js/video_chat_client.js"></script>
</body>
</html>

注意我在最后包含的 js 文件 video_chat_client.js ，这个是我们最主要的代码文件，包含了大部分的逻辑

3. 编写视频聊天的客户端代码，主要就是创建 PeerConnection, 交换 SDP, 通过ICE检查连通性后进行通信

'use strict';

const startButton = document.getElementById('startButton');
const stopButton = document.getElementById('stopButton');
const openButton = document.getElementById('openButton');
const closeButton = document.getElementById('closeButton');

stopButton.disabled = true;
closeButton.disabled = true;


startButton.addEventListener('click', startMedia);
stopButton.addEventListener('click', closeConnection);
openButton.addEventListener('click', join);
closeButton.addEventListener('click', hangup);

// Should use navigator.mediaDevices.getUserMedia of webrtc adapter

// Clean-up function:
// collect garbage before unloading browser's window
window.onbeforeunload = function(e){
    hangup();
}

// Data channel information
var sendChannel, receiveChannel;
var sendButton = document.getElementById("sendButton");
var sendTextarea = document.getElementById("dataChannelSend");
var receiveTextarea = document.getElementById("dataChannelReceive");

// HTML5 <video> elements
var localVideo = document.querySelector('#localVideo');
var remoteVideo = document.querySelector('#remoteVideo');

// Handler associated with 'Send' button
sendButton.onclick = sendData;

// Flags...
var isChannelReady = false;
var isInitiator = false;
var isStarted  = false;

// WebRTC data structures
// Streams
var localStream;
var remoteStream;

// Peer Connection
var pc;

// Peer Connection ICE protocol configuration (either Firefox or Chrome)
var pc_config = webrtcDetectedBrowser === 'firefox' ?
  {'iceServers':[{'urls':'stun:23.21.150.121'}]} : // IP address
  {'iceServers': [{'urls': 'stun:stun.l.google.com:19302'}]};

// Peer Connection contraints: (i) use DTLS; (ii) use data channel
var pc_constraints = {
  'optional': [
    {'DtlsSrtpKeyAgreement': true},
    {'RtpDataChannels': true}
  ]};

// Session Description Protocol constraints:
// - use both audio and video regardless of what devices are available
//var sdpConstraints = {'mandatory': {
//  'OfferToReceiveAudio':true,
//  'OfferToReceiveVideo':true }};

var sdpConstraints = webrtcDetectedBrowser === 'firefox' ?
        {'offerToReceiveAudio':true,'offerToReceiveVideo':true } :
        {'mandatory': {'OfferToReceiveAudio':true, 'OfferToReceiveVideo':true }};


/////////////////////////////////////////////

// Set getUserMedia constraints
const constraints = {
  audio: false,
  video: true
};


var socket = io.connect();

// From this point on, execution proceeds based on asynchronous events...
async function join() {
  // Connect to signalling server
  var room = document.getElementById("roomName").value;

  // Send 'Create or join' message to singnalling server
  if (room !== '') {
    weblog('Create or join room ' + room);
    socket.emit('create or join', room);
  }
}
/////////////////////////////////////////////
async function startMedia() {
  try {
    // Call getUserMedia()
    weblog('Getting user media with constraints: '+ JSON.stringify(constraints));
    const stream = await navigator.mediaDevices.getUserMedia(constraints);
    handleUserMedia(stream);
  } catch (ex) {
    handleUserMediaError(ex);
  }
}
// getUserMedia() handlers...
/////////////////////////////////////////////
function handleUserMedia(stream) {
    localStream = stream;
    attachMediaStream(localVideo, stream);
    weblog('Adding local stream.');
    startButton.disabled = true;
    stopButton.disabled = false;
    sendMessage('got user media');
    if (isInitiator) {
        checkAndStart();
    }
}

function handleUserMediaError(error){
    weblog('Erroe getUserMedia error: ' +  error);
}

// Server-mediated message exchanging...
// 1. Server-->Client...

// Handle 'created' message coming back from server:
// this peer is the initiator
socket.on('created', function (room){
  weblog('Created room ' + room);
  isInitiator = true;

  startMedia();

  checkAndStart();
});

// Handle 'full' message coming back from server:
// this peer arrived too late :-(
socket.on('full', function (room){
  weblog('Room ' + room + ' is full');
});

// Handle 'join' message coming back from server:
// another peer is joining the channel
socket.on('join', function (room){
  weblog('onJoin - another peer made a request to join room ' + room);
  weblog('This peer is the initiator of room ' + room + '!');
  isChannelReady = true;
});

// Handle 'joined' message coming back from server:
// this is the second peer joining the channel
socket.on('joined', function (room){
  weblog('onJoined - this peer has joined room ' + room);
  isChannelReady = true;

  startMedia();
});

// Server-sent log message...
socket.on('log', function (array){
  console.log.apply(console, array);
});

// Receive message from the other peer via the signalling server
socket.on('message', function (message){
  weblog('onMessage: Received message:' +  JSON.stringify(message));
  if (message === 'got user media') {
        checkAndStart();
  } else if (message.type === 'offer') {
    if (!isInitiator && !isStarted) {
      checkAndStart();
    }
    pc.setRemoteDescription(new RTCSessionDescription(message));
    doAnswer();
  } else if (message.type === 'answer' && isStarted) {
    pc.setRemoteDescription(new RTCSessionDescription(message));
  } else if (message.type === 'candidate' && isStarted) {
    var candidate = new RTCIceCandidate({sdpMLineIndex:message.label,
      candidate:message.candidate});
    pc.addIceCandidate(candidate);
  } else if (message === 'bye' && isStarted) {
    handleRemoteHangup();
  }
});
////////////////////////////////////////////////

// 2. Client-->Server
////////////////////////////////////////////////
// Send message to the other peer via the signalling server
function sendMessage(message){
  weblog('Sending message: ' + message);
  socket.emit('message', message);
}
////////////////////////////////////////////////////

////////////////////////////////////////////////////
// Channel negotiation trigger function
function checkAndStart() {
  weblog('checkAndStart: isStarted='+ isStarted + ", isChannelReady=" +  isChannelReady);
  if (!isStarted && typeof localStream != 'undefined' && isChannelReady) {
    createPeerConnection();
    pc.addStream(localStream);
    isStarted = true;
    if (isInitiator) {
      doCall();
    }
  }
}

/////////////////////////////////////////////////////////
// Peer Connection management...
function createPeerConnection() {
  try {
    pc = new RTCPeerConnection(pc_config, pc_constraints);
    pc.onicecandidate = handleIceCandidate;
    weblog('Created RTCPeerConnnection with:\n' +
      '  config: \'' + JSON.stringify(pc_config) + '\';\n' +
      '  constraints: \'' + JSON.stringify(pc_constraints) + '\'.');
  } catch (e) {
    weblog('Failed to create PeerConnection, exception: ' + e.message);
    alert('Cannot create RTCPeerConnection object.');
      return;
  }
  pc.onaddstream = handleRemoteStreamAdded;
  pc.onremovestream = handleRemoteStreamRemoved;

  if (isInitiator) {
    try {
      // Create a reliable data channel
      sendChannel = pc.createDataChannel("sendDataChannel",
        {reliable: true});
      trace('Created send data channel');
    } catch (e) {
      alert('Failed to create data channel. ');
      trace('createDataChannel() failed with exception: ' + e.message);
    }
    sendChannel.onopen = handleSendChannelStateChange;
    sendChannel.onmessage = handleMessage;
    sendChannel.onclose = handleSendChannelStateChange;
  } else { // Joiner
    pc.ondatachannel = gotReceiveChannel;
  }
}

// Data channel management
function sendData() {
  var data = sendTextarea.value;
  if(isInitiator) sendChannel.send(data);
  else receiveChannel.send(data);
  trace('Sent data: ' + data);
}

// Handlers...

function gotReceiveChannel(event) {
  trace('Receive Channel Callback');
  receiveChannel = event.channel;
  receiveChannel.onmessage = handleMessage;
  receiveChannel.onopen = handleReceiveChannelStateChange;
  receiveChannel.onclose = handleReceiveChannelStateChange;
}

function handleMessage(event) {
  weblog('Received message: ' + event.data);
  receiveTextarea.value += event.data + '\n';
}

function handleSendChannelStateChange() {
  var readyState = sendChannel.readyState;
  weblog('handleSendChannelStateChange, send channel state is: ' + readyState);
  // If channel ready, enable user's input
  if (readyState == "open") {
    dataChannelSend.disabled = false;
    dataChannelSend.focus();
    dataChannelSend.placeholder = "";
    sendButton.disabled = false;
  } else {
    dataChannelSend.disabled = true;
    sendButton.disabled = true;
  }
}

function handleReceiveChannelStateChange() {
  var readyState = receiveChannel.readyState;
  weblog('handleReceiveChannelStateChange: seceive channel state is: ' + readyState);
  // If channel ready, enable user's input
  if (readyState == "open") {
        dataChannelSend.disabled = false;
        dataChannelSend.focus();
        dataChannelSend.placeholder = "";
        sendButton.disabled = false;
      } else {
        dataChannelSend.disabled = true;
        sendButton.disabled = true;
      }
}

// ICE candidates management
function handleIceCandidate(event) {
  weblog('handleIceCandidate event: ' + JSON.stringify( event));
  if (event.candidate) {
    sendMessage({
      type: 'candidate',
      label: event.candidate.sdpMLineIndex,
      id: event.candidate.sdpMid,
      candidate: event.candidate.candidate});
  } else {
    weblog('End of candidates.');
  }
}

// Create Offer
function doCall() {
  weblog('Creating Offer...');
  pc.createOffer(setLocalAndSendMessage, onSignalingError, sdpConstraints);
}

// Signalling error handler
function onSignalingError(error) {
    console.log('Failed to create signaling message : ' + error.name);
}

// Create Answer
function doAnswer() {
  console.log('Sending answer to peer.');
  pc.createAnswer(setLocalAndSendMessage, onSignalingError, sdpConstraints);
}

// Success handler for both createOffer()
// and createAnswer()
function setLocalAndSendMessage(sessionDescription) {
  pc.setLocalDescription(sessionDescription);
  sendMessage(sessionDescription);
}

/////////////////////////////////////////////////////////
// Remote stream handlers...

function handleRemoteStreamAdded(event) {
  weblog('Remote stream added.');
  attachMediaStream(remoteVideo, event.stream);
  remoteStream = event.stream;
}

function handleRemoteStreamRemoved(event) {
  weblog('Remote stream removed.');
  console.log('Remote stream removed. Event: ', event);
}
/////////////////////////////////////////////////////////

/////////////////////////////////////////////////////////
// Clean-up functions...

function hangup() {
  console.log('Hanging up.');
  closeConnection();
  sendMessage('bye');
}

function handleRemoteHangup() {
  console.log('Session terminated.');
  closeConnection();
  isInitiator = false;
}

function closeConnection() {
  isStarted = false;
  if (sendChannel) sendChannel.close();
  if (receiveChannel) receiveChannel.close();
  if (pc) pc.close();
  pc = null;
  sendButton.disabled=true;
}

代码就这么多，下面我们来讲讲怎么运行

4. 启动并运行视频聊天程序

你需要准备至少两台电脑，在一台电脑上用 nodejs 启动 Web 服务器

完整源码参见 https://gitee.com/walterfan/webrtc_primer/blob/main/examples/video_chat_server.js

npm installl
# 这个命令等同于 node video_chart_server.js
npm start

注意WebRTC 强制使用 HTTPS, 我用了自签名的证书，你需要忽略安全警告， Chrome浏览器可以通过
chrome://flags/#allow-insecure-localhost 来关闭警告。

启动后，假设你的 web 服务器IP 是 192.168.1.28，

在一台电脑上打开浏览器，输入地址 https://192.168.1.2:8183，再输入房间名称 "demo"，点击 "Join" 按钮。
在另一台电脑上重复以上操作

最终效果如下, 可以在两台电脑上进行视频和文字聊天

借助 WebRTC 强大的 API , 需要写的代码也没多少，我在客户端和服务端都记录了详细的日志供您参考

参考资料

<<Real-Time Communication with WebRTC>>
https://webrtc.github.io/samples/

如何创建 WebRTC 会话二

WebRTC 的信令，媒体以及数据通道

会话创建过程(JSEP) 与两个最重要的状态机

1. 我们要写一个上面提到的信令服务器, 同时也作为一个网页服务器

第一步创建 package.json

第二步：创建 video_chat_server.js

第三步：引入 socket.io 库，添加对于客户端连接的管理

2. 写一个支持视频和文本聊天的网页

3. 编写视频聊天的客户端代码，主要就是创建 PeerConnection, 交换 SDP, 通过ICE检查连通性后进行通信

4. 启动并运行视频聊天程序

参考资料

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读

音视频开发笔记

如何创建 WebRTC 会话二

WebRTC 的信令，媒体以及数据通道

会话创建过程(JSEP) 与两个最重要的状态机

1. 我们要写一个上面提到的信令服务器, 同时也作为一个网页服务器

第一步创建 package.json

第二步： 创建 video_chat_server.js

第三步：引入 socket.io 库，添加对于客户端连接的管理

2. 写一个支持视频和文本聊天的网页

3. 编写视频聊天的客户端代码，主要就是创建 PeerConnection, 交换 SDP, 通过ICE检查连通性后进行通信

4. 启动并运行视频聊天程序

参考资料

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读

音视频开发笔记

第二步：创建 video_chat_server.js