Camera2 教程 4：通过TEXTURE OES方式实现相机

作者: 古风子 | 来源:发表于2020-03-25 21:17 被阅读0次

青橙相机

GLES11Ext.GL_TEXTURE_EXTERNAL_OES的用处是什么？
上一张实现相机预览的每一帧的输出格式是YUV的（YUV），那么这个扩展纹理的作用就是实现YUV格式到RGB的自动转化，我们就不需要再为此写YUV转RGB的代码了

1. 创建xml和Render

//xml
<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <com.jdf.camera.ui.CameraV2GLSurfaceView
        android:id="@+id/glsurfaceView"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

</RelativeLayout>

//Activity
   @Override
    protected void onCreate(@Nullable Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_camera2_oes);
        //获取SurfaceView
        glSurfaceView = findViewById(R.id.glsurfaceView);
        //封装相机操作
        mCameraLoader = new Camera2OESLoader(this);
        //主要是实例化Render，并绑定
        glSurfaceView.init(mCameraLoader, false, Camera2OESActivity.this);
        findViewById(R.id.saveImage).setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                ImageUtils.saveBitmap = true;
            }
        });
    }

//GlSurfaceView
public class CameraV2GLSurfaceView extends GLSurfaceView {
    public static final String TAG = "Filter_CameraV2GLSurfaceView";
    private JImageRender mCameraV2Renderer;

    public void init(Camera2OESLoader camera, boolean isPreviewStarted, Context context) {
        setEGLContextClientVersion(2);

        mCameraV2Renderer = new JImageRender(new JImageFilter(context));
        mCameraV2Renderer.init(this, camera, isPreviewStarted, context);

        setRenderer(mCameraV2Renderer);
    }

    public CameraV2GLSurfaceView(Context context, AttributeSet attributes) {
        super(context, attributes);
    }
}

2 开启相机

在Activity的onResume中，判断glSurfaceView加载完毕后，获取相机相机传感器方向，并且根据GlSurfaceView大小获取合适的预览大小

//Activity
    @Override
    protected void onResume() {
        super.onResume();
        boolean laidOut = ViewCompat.isLaidOut(glSurfaceView);
        boolean b = !glSurfaceView.isLayoutRequested();
        JLog.d(TAG, "onResume.....laidOut[%b],[%b]",laidOut,b);
        if (laidOut && b) {
            mCameraLoader.onResume(glSurfaceView.getWidth(), glSurfaceView.getHeight());
        } else {
            glSurfaceView.addOnLayoutChangeListener(new View.OnLayoutChangeListener() {
                @Override
                public void onLayoutChange(View v, int left, int top, int right, int bottom, int oldLeft, int oldTop,
                                           int oldRight, int oldBottom) {
                    JLog.d(TAG, "onResume.....onLayoutChange");
                    glSurfaceView.removeOnLayoutChangeListener(this);
                    mCameraLoader.onResume(glSurfaceView.getWidth(), glSurfaceView.getHeight());
                }
            });
        }
    }

Camera2OESLoader.java

    public void onResume(int width, int height) {
        JLog.d(TAG, "onResume[%d,%d]...", width, height);
        mViewWidth = width;
        mViewHeight = height;
        setUpCamera();
    }


    protected void setUpCamera() {
        try {
            //获取屏幕方向，相机传感器方向，和预览大小
            setUpCameraOutputs();
            if (ActivityCompat.checkSelfPermission(mActivity, Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED) {
                JLog.e(TAG, "Dont have CAMERA permission");
                return;
            }
            mCameraManager.openCamera(mCameraId, mCameraDeviceCallback, mCameraHandler);
        } catch (CameraAccessException e) {
            JLog.e(TAG, "Opening camera (ID: " + mCameraId + ") failed.");
            e.printStackTrace();
        }
    }

mCameraHandler是根据Camera2OESLoader中HandlerThread创建的，因为后续我们需要再Render线程中启动预览，为了保证相机打开和启动预览在同一个线程，需要指定同一个Handler

相机启动成功后，我们去Render中创建OES纹理id，SufaceTexture和启动预览

3 启动预览

JImageRender.onSurfaceCreated

创建OES 纹理id

   @Override
    public void onSurfaceCreated(GL10 gl, EGLConfig config) {
        JLog.i(TAG, "onSurfaceCreated......");
        mOESTextureId = Utils.createOESTextureObject();
        mFilterEngine.ifNeedInit();

    }

JImageRender.onDrawFrame
然后在onDrawFrame第一次调用时，完成SurfaceTexture初始化和预览启动

   @Override
    public void onDrawFrame(GL10 gl) {

        Long t1 = System.currentTimeMillis();
        if (mSurfaceTexture != null) {
            mSurfaceTexture.updateTexImage();
            mSurfaceTexture.getTransformMatrix(transformMatrix);
        }

        if (!bIsPreviewStarted) {
            bIsPreviewStarted = initSurfaceTexture();
            bIsPreviewStarted = true;
            return;
        }

        glClearColor(1.0f, 0.0f, 0.0f, 0.0f);

        mFilterEngine.onDraw(transformMatrix,mOESTextureId,glCubeBuffer,glTextureBuffer);

        long t2 = System.currentTimeMillis();
        long t = t2 - t1;
        Log.i(TAG, "onDrawFrame: time: " + t);
    }

JImageRender. initSurfaceTexture

  public boolean initSurfaceTexture() {
     
        mSurfaceTexture = new SurfaceTexture(mOESTextureId);
        mCameraLoaer.setPreviewTexture(mSurfaceTexture);
        mCameraLoaer.startPreview();
        return true;
    }

Camera2OESLoader.startPreview

 public void startPreview() {
       
        mSurfaceTexture.setDefaultBufferSize(mPreviewSize.getWidth(), mPreviewSize.getHeight());
        final Surface surface = new Surface(mSurfaceTexture);

        try {

            mCameraDevice.createCaptureSession(Arrays.asList( surface), new CameraCaptureSession.StateCallback() {
                @Override
                public void onConfigured(@NonNull CameraCaptureSession session) {
                    try {
                        CaptureRequest.Builder builder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
                        builder.addTarget(surface);
                        mCaptureRequest = builder.build();
                        mCameraCaptureSession = session;
                        mCameraCaptureSession.setRepeatingRequest(mCaptureRequest, null, mCameraHandler);
                    } catch (CameraAccessException e) {
                        e.printStackTrace();
                    }
                }

                @Override
                public void onConfigureFailed(@NonNull CameraCaptureSession session) {

                }
            }, mCameraHandler);
        } catch (CameraAccessException e) {
            e.printStackTrace();
        }
    }

4 绘制渲染

预览启动后成功后，会更新SurfaceTexture，并更新纹理数据对应的纹理举证，然后将该SurfaceTexture对应的OES纹理ID传递到OPENGL进行绘制

    @Override
    public void onDrawFrame(GL10 gl) {

        Long t1 = System.currentTimeMillis();
        if (mSurfaceTexture != null) {
           //更新SurfaceTexture纹理内容
            mSurfaceTexture.updateTexImage();
            mSurfaceTexture.getTransformMatrix(transformMatrix);
        }

        if (!bIsPreviewStarted) {
            bIsPreviewStarted = initSurfaceTexture();
            bIsPreviewStarted = true;
            return;
        }

        glClearColor(1.0f, 0.0f, 0.0f, 0.0f);
        //使用opengl绘制mOESTextureId对应的纹理内容
        mFilterEngine.onDraw(transformMatrix, mOESTextureId,glCubeBuffer,glTextureBuffer)
  
    }

注意：
updateTexImage()方法只能在包OpenGLES环境的线程里调用，即Renderer接口所独立创建的线程当中。一般在onDrawFrame中调用updateTexImage()将数据绑定给OpenGLES对应的纹理对象

JimageFilter.onDraw

  public void onDraw(float[] transformMatrix, int textureId, FloatBuffer cubeBuffer, FloatBuffer textureBuffer) {
      
        GLES30.glUseProgram(glProgId);
           ....
        if (textureId != OpenGlUtils.NO_TEXTURE) {
            ...
           glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, textureId);
           ...
           glUniformMatrix4fv(uTextureMatrixLocation, 1, false, transformMatrix, 0);
        }
         ...
        GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, 0);

    }

onDraw过程与上一节的相机预览流程一致，主要差别有两点

绑定和解绑纹理id的类型为GL_TEXTURE_EXTERNAL_OES
需要额外将从SurfaceTexture中产生的变换矩阵uTextureMatrixLocation传递到着色器中

5 定义OES着色器

因为纹理id使用的是OES格式，着色器中纹理类型也要对应修改，相对于上以上相机预览实现，作色器主要有两个变化

顶点着色器中，需要传递SurfaceTexture产生的变换矩阵对象uTextureMatrix，并根据矩阵值调整纹理坐标位置

attribute vec4 position;
uniform mat4 uTextureMatrix;
attribute vec4 inputTextureCoordinate;
varying vec2 textureCoordinate;
void main()
{
  textureCoordinate = (uTextureMatrix * inputTextureCoordinate).xy;
  gl_Position = position;
}`

片元着色器中，需要声明纹理的类型为samplerExternalOES ，而不是sampler2D; 需要在文件头声明使用了samplerExternalOES，否则加载着色器会报错

#extension GL_OES_EGL_image_external : require
precision mediump float;
uniform samplerExternalOES inputImageTexture;
varying vec2 textureCoordinate;
void main()
{
  gl_FragColor = texture2D(inputImageTexture, textureCoordinate);
}

6 显示效果

预览效果

7 流程总结

相机OES预览.png

主要流程为：

定义GlSurfaceView和定义OEG纹理类型的着色器
将GlSurfaceView和Render,设置渲染模式
在GlSurfaceView加载完毕后，启动相机
在Render的onSurfaceCreate中，完成OES纹理id创建和初始化Opengl对象
在相机开启成功的时候，在onDrawFrame中完成预览启动；之所以在onDrawFrame启动，是因为其他阶段，相机不一定启动成功
预览启动成功后，会在Render的每一帧调用时，更新SurfaceTexture的内容，并更新对应的OES纹理id；将纹理id传递到opengl对象中进行渲染
渲染成功后，预览数据会显示到屏幕上

8 代码位置

代码具体实现参考 QCCamera中的JCamera2OESFboActivity

Camera2 教程 4：通过TEXTURE OES方式实现相机

1. 创建xml和Render

2 开启相机

3 启动预览

4 绘制渲染

5 定义OES着色器

6 显示效果

7 流程总结

8 代码位置

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读

影视处理