WebRTC之摄像头预览

作者: FlyerGo | 来源:发表于2021-07-02 09:15 被阅读0次

在前面《WebRTC之服务器搭建》我们已经搭建好了WebRTC所需的服务器环境，主要是三个服务器：
房间服务器、信令服务器以及TURN穿透服务器。

后续我们将学习如何使用WebRTC一步一步实现音视频通话。今天我们将学习如何使用WebRTC预览摄像头数据。

这里透个底，后面的学习过程中大部分的实践都是基于WebRTC的官方封装库，因此绝大部分的代码都是Java或者Kotlin，暂时不会涉及到JNI的相关代码，所以门槛还是非常低的。

Good good study，day day up. So easy...

引入依赖库

首先我们在Android Studio工程中引入WebRTC的依赖库:

 implementation 'org.webrtc:google-webrtc:1.0.+'

动态权限

首先肯定是需要CAMERA权限的，如果需要音频数据则还需要RECORD_AUDIO权限。

对于动态权限相信有Android开发基础的童鞋们都不陌生了，gitHub上也有很多相关的开源库，笔者在这里就不多做介绍了。

预览摄像头

WebRTC作为一个点对点通信完整的解决方案，对于摄像头数据的获取及预览都已经做好了完整封装，开发者直接调用相关的API即可，
并不需要开发者编写OpenGL纹理渲染等相关的逻辑代码。

如果童鞋们对如何使用OpenGL渲染摄像头数据感兴趣的话可以参考笔者之前的文章：《使用OpenGL预览CameraX摄像头数据》

使用WebRTC预览摄像头数据主要有以下几个步骤：

1、创建EglBase及SurfaceViewRenderer

其中EglBase一个重要的功能就是提供EGL的渲染上下文及EGL的版本兼容。

SurfaceViewRenderer则是一个继承于SurfaceView的渲染View，提供了OpenGL渲染图像数据的功能。

// 创建EglBase
rootEglBase = EglBase.create()
camera_preview.init(rootEglBase?.eglBaseContext, null)
//硬件加速
camera_preview.setEnableHardwareScaler(true)
camera_preview.setScalingType(RendererCommon.ScalingType.SCALE_ASPECT_FILL)

2、创建VideoCapturer

VideoCapturer主要作用则是提供摄像头数据，比如控制分辨率以及帧率等。

通过CameraEnumerator接口兼容Camera1和Camera2：

   // 创建VideoCapturer
    private fun createVideoCapture(): VideoCapturer? {
        return if (Camera2Enumerator.isSupported(this)) {
            createCameraCapture(Camera2Enumerator(this))
        } else {
            createCameraCapture(Camera1Enumerator(true))
        }
    }


   //  真正创建VideoCapturer的实现
    private fun createCameraCapture(enumerator: CameraEnumerator): VideoCapturer? {
        val deviceNames = enumerator.deviceNames
        // First, try to find front facing camera
        for (deviceName in deviceNames) {
            if (enumerator.isFrontFacing(deviceName)) {

                val videoCapture: VideoCapturer? = enumerator.createCapturer(deviceName, null)
                if (videoCapture != null) {
                    return videoCapture
                }
            }
        }

        for (deviceName in deviceNames) {
            if (!enumerator.isFrontFacing(deviceName)) {
                val videoCapture: VideoCapturer? = enumerator.createCapturer(deviceName, null)
                if (videoCapture != null) {
                    return videoCapture
                }
            }
        }
        return null
    }

VideoCapturer创建完成之后需要使用SurfaceTextureHelper进行初始化，否则调用预览的时候会抛出未初始化的异常：

  // 初始化
        mSurfaceTextureHelper =
            SurfaceTextureHelper.create("CaptureThread", rootEglBase?.eglBaseContext)
        // 创建VideoSource
        val videoSource = mPeerConnectionFactory!!.createVideoSource(false)
        mVideoCapture?.initialize(
            mSurfaceTextureHelper,
            applicationContext,
            videoSource.capturerObserver
        )


    /**
     * 创建PeerConnectionFactory
     */
    private fun createPeerConnectionFactory(context: Context?): PeerConnectionFactory? {
        val encoderFactory: VideoEncoderFactory
        val decoderFactory: VideoDecoderFactory
        encoderFactory = DefaultVideoEncoderFactory(
            rootEglBase?.eglBaseContext,
            false /* enableIntelVp8Encoder */,
            true
        )
        decoderFactory = DefaultVideoDecoderFactory(rootEglBase?.eglBaseContext)
        PeerConnectionFactory.initialize(
            PeerConnectionFactory.InitializationOptions.builder(context)
                .setEnableInternalTracer(true)
                .createInitializationOptions()
        )
        val builder = PeerConnectionFactory.builder()
            .setVideoEncoderFactory(encoderFactory)
            .setVideoDecoderFactory(decoderFactory)
        builder.setOptions(null)
        return builder.createPeerConnectionFactory()
    }

3、创建VideoTrack

VideoTrack是视频轨道，类似的还有AudioTrack音频轨道，它的作用将VideoCapturer获取到的视频数据结合VideoSource输出到SurfaceViewRenderer渲染显示。

val VIDEO_TRACK_ID = "1" //"ARDAMSv0"
// 创建VideoTrack
 mVideoTrack = mPeerConnectionFactory!!.createVideoTrack(VIDEO_TRACK_ID,
            videoSource
        )
        mVideoTrack?.setEnabled(true)

      // 绑定渲染View
        mVideoTrack?.addSink(camera_preview)

使用VideoCapturer开启渲染

在Activity的相关生命周期中开启预览即可：

   override fun onResume() {
        super.onResume()
        // 开启摄像头预览
        mVideoCapture?.startCapture(
            VIDEO_RESOLUTION_WIDTH,
            VIDEO_RESOLUTION_HEIGHT,
            VIDEO_FPS
        )
    }

完整代码

CapturePreviewActivity.kt:

package com.fly.webrtcandroid

import android.content.Context
import androidx.appcompat.app.AppCompatActivity
import android.os.Bundle
import org.webrtc.*

/**
 * 摄像头预览
 */
class CapturePreviewActivity : AppCompatActivity() {

    val VIDEO_TRACK_ID = "1" //"ARDAMSv0"

    private val VIDEO_RESOLUTION_WIDTH = 1280
    private val VIDEO_RESOLUTION_HEIGHT = 720
    private val VIDEO_FPS = 30


    //    绘制全局的上下文
    private var rootEglBase: EglBase? = null
    private var mVideoTrack: VideoTrack? = null

    private var mPeerConnectionFactory: PeerConnectionFactory? = null

    //纹理渲染
    private var mSurfaceTextureHelper: SurfaceTextureHelper? = null

    private var mVideoCapture: VideoCapturer? = null

    private val camera_preview by lazy {
        findViewById<SurfaceViewRenderer>(R.id.camera_preview)
    }

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_capture_preview)

        rootEglBase = EglBase.create()

        camera_preview.init(rootEglBase?.eglBaseContext, null)

        //悬浮顶端
        camera_preview.setZOrderMediaOverlay(true)
        //硬件加速
        camera_preview.setEnableHardwareScaler(true)
        camera_preview.setScalingType(RendererCommon.ScalingType.SCALE_ASPECT_FILL)

        mPeerConnectionFactory = createPeerConnectionFactory(this)

        mVideoCapture = createVideoCapture()

        // 初始化
        mSurfaceTextureHelper =
            SurfaceTextureHelper.create("CaptureThread", rootEglBase?.eglBaseContext)
        // 创建VideoSource
        val videoSource = mPeerConnectionFactory!!.createVideoSource(false)
        mVideoCapture?.initialize(
            mSurfaceTextureHelper,
            applicationContext,
            videoSource.capturerObserver
        )

        mVideoTrack = mPeerConnectionFactory!!.createVideoTrack(VIDEO_TRACK_ID,
            videoSource
        )
        mVideoTrack?.setEnabled(true)
        mVideoTrack?.addSink(camera_preview)

    }

    /**
     * 创建PeerConnectionFactory
     */
    private fun createPeerConnectionFactory(context: Context?): PeerConnectionFactory? {
        val encoderFactory: VideoEncoderFactory
        val decoderFactory: VideoDecoderFactory
        encoderFactory = DefaultVideoEncoderFactory(
            rootEglBase?.eglBaseContext,
            false /* enableIntelVp8Encoder */,
            true
        )
        decoderFactory = DefaultVideoDecoderFactory(rootEglBase?.eglBaseContext)
        PeerConnectionFactory.initialize(
            PeerConnectionFactory.InitializationOptions.builder(context)
                .setEnableInternalTracer(true)
                .createInitializationOptions()
        )
        val builder = PeerConnectionFactory.builder()
            .setVideoEncoderFactory(encoderFactory)
            .setVideoDecoderFactory(decoderFactory)
        builder.setOptions(null)
        return builder.createPeerConnectionFactory()
    }

    private fun createVideoCapture(): VideoCapturer? {
        return if (Camera2Enumerator.isSupported(this)) {
            createCameraCapture(Camera2Enumerator(this))
        } else {
            createCameraCapture(Camera1Enumerator(true))
        }
    }

    private fun createCameraCapture(enumerator: CameraEnumerator): VideoCapturer? {
        val deviceNames = enumerator.deviceNames
        // First, try to find front facing camera
        for (deviceName in deviceNames) {
            if (enumerator.isFrontFacing(deviceName)) {

                val videoCapture: VideoCapturer? = enumerator.createCapturer(deviceName, null)
                if (videoCapture != null) {
                    return videoCapture
                }
            }
        }

        for (deviceName in deviceNames) {
            if (!enumerator.isFrontFacing(deviceName)) {
                val videoCapture: VideoCapturer? = enumerator.createCapturer(deviceName, null)
                if (videoCapture != null) {
                    return videoCapture
                }
            }
        }
        return null
    }

    override fun onResume() {
        super.onResume()
        // 开启摄像头预览
        mVideoCapture?.startCapture(
            VIDEO_RESOLUTION_WIDTH,
            VIDEO_RESOLUTION_HEIGHT,
            VIDEO_FPS
        )
    }

}

布局文件activity_capture_preview.xml:

<?xml version="1.0" encoding="utf-8"?>
<org.webrtc.SurfaceViewRenderer xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:id="@+id/camera_preview"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".CapturePreviewActivity">

</org.webrtc.SurfaceViewRenderer>

关注我，一起进步，人生不止coding！！！

WebRTC之摄像头预览

引入依赖库

动态权限

预览摄像头

1、创建EglBase及SurfaceViewRenderer

2、创建VideoCapturer

3、创建VideoTrack

使用VideoCapturer开启渲染

完整代码

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读