美文网首页
Android VideoPlayer 调用的OpenGL ES

Android VideoPlayer 调用的OpenGL ES

作者: 小老鼠oo | 来源:发表于2019-11-25 18:18 被阅读0次

    这次主要是为了看我们的OGLES driver里为什么会malloc的比mali多,dumpheap发现排名靠前的都是与texture有关的,我记得是glTexImage2D和glTexSubImage。里面是为了Mipmap临时malloc了system memory,但是谁知道后面会不会再用?不敢轻易free。OGL ES API我也不熟,怎么也得系统的看看。

    • 描述一下接口层的定义,spec和书籍会有
    • 再看driver code里重点实现。

    书籍参考OpenGL ES 3.0 Programming Guide, www.java.1234.com可以下载,电子版,带目录。

    这本书的中文版有点问题,他翻译的太多了,且不给英文原文,违和感很强。比如在书里看了统一变量,上下文,片段着色器,绘图表面的知识,再看代码,却很难在看到Uniform,context,fragment shader,surface时找到共鸣。

    直接看英文书又太难,还是先看了中文熟悉基本的概念,再看英文,好看一点。


    EGL API序列:

    前面的API都是给状态,状态保存在glContext,只有Draw才会真正让GPU做事情,所有的draw API都会走到DrawPrimitive(),后面的Begin() => ValidateAttribute()可以看到所有的过程:

    • Validate Input Assembler (IA)
    • Validate Vertex Shader (VS)
    • Validate Rasterization (RS)
    • ValidateZlxStage1 (ZL1)
    • Validate pixel shader (PS)
    • Vlidate output merger (OM)
    • Validate ZLx stage2 (ZL2)
      上述比如vbInfo,都是保存在glContext里面的

    开始是EGL调用,EGL提供了下面的机制:

    • communication with native windowing system
    • query available configurations of drawing surfaces
    • create drawing surface
    • managing rendering resources such as texture maps

    eglGetDisplay(EGL_DEFAULT_DISPLAY)=display_0;

    • API : EGLDisplay eglGetDisplay(EGLNativeDisplayType displayId)
      its open a connection to EGL display server, EGLNativeDisplayType is defined to match the native window system's display type.
    • Dirver:
      EGLNativeDisplayType就是typedef int,EGL_DEFAULT_DISPLAY就是((EGLNativeDisplayType)0),而已。
      driver会创建一个_EGLdisplay *pDpy,return的是pDpy->hDpy,也就是一个handle。
      新创建的pDpy添加到了DIsplayList,这是一个__eglGlobal.pDpyList,上面的handle也在__eglGlobal维护的.
      eglGlobal是一个全局的结构体变量。在attachProcess() => __eglInitGlobals() 时被init,这个函数应该是整个EGL.so第一个被触发的地方,谁触发的?
      这里写的有问题,最早的应该是InitDriver()而不是attachProcess(),进而__glDpInitialization() => InitDriver()
      后续的egl函数都要传一个display过去。

    eglInitialize(display_0, NULL, NULL);

    • API: EGLBoolean eglInitialize(EGLDisplay display, EGLint *majorVertion, EGLint *minorVertsion)
      display : EGL display connection
      这个函数是init EGL's internal data structures
    • driver:
      前面说过,eglGlobal维护了pDpy的链表,现在按照hDpy找到它并取出来。
      这里做的事情不少,init configs tables, load es1/2 driver。
      这里还填了dDpy->esDriverAPI[__EGL_ES_VETSION_2_X].createDisplay的函数指针,然后调用其createDisplay(),返回值保存在pDpy->esPrivateData[2_X],这里的函数指针都是eglXXX。也就是eglXXX前面多了下划线
      在eglCreateDisplay(),创建了_EGLdisplayPrivate,返回后保存在pDpy->esPrivateData[2_x]里面,

    DEFINE POINTER name=attrib_list_0, type=egl_attrib_list, count=0, data=imm{EGL_RENDERABLE_TYPE=EGL_OPENGL_ES2_BIT, EGL_RED_SIZE=8, EGL_GREEN_SIZE=8, EGL_BLUE_SIZE=8, EGL_ALPHA_SIZE=8, EGL_DEPTH_SIZE=0, EGL_CONFIG_CAVEAT=EGL_NONE, EGL_STENCIL_SIZE=8, EGL_SURFACE_TYPE=EGL_WINDOW_BIT, EGL_NONE};
    eglChooseConfig(display_0, attrib_list_0, configs_0, 1, pointer_0);

    • API: 通过提供的attrib_list,得到1个合适的EGLConfig
      可以看到,AP需要的是 RGBA8888的rendering surface,没有depth buffer, 有stencil buffer。需要的surface type是window。
    • driver:
      上一步已经创建好了pDpy->pConfigList,与传入的attri_list作比较,返回合适的。

    eglCreateContext(display_0, configs_0[0], EGL_NO_CONTEXT, attrib_list_1)=context_0; //0x3

    • API: rendering context is a data structure internal to OGLES that contains all of state info required for operation.for example, it contains references to vertex and fragment shaders and array of vertex data.
      context保存了所有状态,比如vertex,fragment shaders, vertex数组。
    • Driver:
      重要的部分还是在gl里面,egl层只是保存了个pCtx->privateData,gl下glContext保存在其pCtx->privateData。

    DEFINE POINTER name=attrib_list_2, type=egl_attrib_list, count=0, data=imm{EGL_WIDTH=1, EGL_HEIGHT=1, EGL_NONE};
    eglCreatePbufferSurface(display_0, configs_0[0], attrib_list_2)=surface_0;

    • API:you can render into nonvisible off-screen surface called pbuffers(pixel buffer). is often used for generating texture maps 纹理贴图. if render to a texture we recommend using frambuffer objects instead of pbuffers because they are more efficient.
      但是pbuffer仍然有用,比如在OES做offscreen surface render然后再OpenVG等其他API用这个texture.
      这里给的attrib_list的宽高是1,得到一个surface_0
    • Driver:
      driver中surface对应的是EGLdrawable *pDraw,同样保存在eglGlobal里,其链表叫pDrawList。同样调用gl的__eglCreateDrawable(),把gl创建的Drawable保存在pDraw->esPrivateData.
      gl中创建的是EGLdrwablePrivate *pDraw,他似乎只是个外壳,_GLdrawablePrivate *glPriv才是重要的,最重要的是通过Elte层的createDrawable创建的drawable保存在了glPriv->dp.privateData,dp是GLdrawablePrivateRec,赋值了很多函数指针,其中有一个是_glUpdateDrawableAllocation(glPriv, glPriv->da),通过他create_renderbuffer真正的创建 buffer,不过只有置了PBUFFER_BIT才会在这个时候创建,正常都是在__glNotifyChangeDrawableSize()时才做的。

    eglMakeCurrent(display_0, surface_0, surface_0, context_0);

    • API:
      AP may create multiple EGLContext for various purposes, we need a way to associate a EGLContext with rendering surface : make current
    • Driver:
      做EGLcontext与EGLdrawable的绑定,这里会调用gl层的__eglUpdateDrawable(),比较的是glPriv与da的宽高等是否相等。glPriv的宽高是在createDrawable时拿的,da的宽高是在updateDrawableAllocation()时拿的,也就是说,MakeCurrent时重新同步的一下宽高等信息,平时NotifyChangeDrawableSize()也会更新。记得,glPriv就是_GLdrawablePrivate

    这里的代码现在看也觉得很乱,后面再补充。至此,EGL调用做完了,本地环境有了,开始调用GL API.


    GL ES API序列:

    Vertex:

    DEFINE POINTER name=buffer_0, type=uint, count=1, data=imm{null};
    glGenBuffers(1, buffer_0);
    API : its about Vertex Buffer Object
    通过vertex arrays指定的vertex data保存在client memory,在调用glDrawArrays/DrawElements时必须保证这些data从client memory复制到了graphics memory。但是,不需要每次draw call都copy vertex data的,一直cache data in graphics memory更好。这就是vertex buffer objects的目的。
    Vertex buffer objects 允许AP从graphics memory去allocate和cache vertex data,并从这个memory 去render,从而避免每次画primitive时都要resending data。
    不只是vertex data, 描述primitive vertex indices的element indices,作为glDrawElements的arg,也可以cached在这个memory。
    OGLES 3.0支持两种buffer object:

    • vertex : array buffer objects, GL_ARRAY_BUFFER,store vertex data.
    • primitive data: element array buffer objects,GL_ELEMENT_ARRAY_BUFFER,store indices of a primitive.

    glBindBuffer(GL_ARRAY_BUFFER, buffer_0[0]);

    DEFINE POINTER name=buffer_1, type=ubyte, count=64, file_offset=0, data=file{combine_0.bin};
    glBufferData(GL_ARRAY_BUFFER, 64, buffer_1, GL_STATIC_DRAW);

    • API: data argu can be NULL, indicating that reserved data store remains uninitialized.
      if data is valid pointer, contens of data are copied to allocated data store.
    • Driver:
      首先要拿到__GLbufferObject bufObj,根据其targetIndex(如ARRAY_BUFFER_INDEX, ELEMENT_ARRAY_BUFFER_INDEX),从gc->bufferObject.generalBindingPoint[index].boundBufObj拿,这个变量应该是在glBindBuffer的时候赋值的。
      得到BufObj和
      data,在Elite层的BufferData => V ertexBufferUpload()里做blt,src自然是data,因为他来自system memory,所以要填到createData.pPreAllocPackedInfo.pSysMem,然后为其rmCreateResource()。dst就是bufobj->VertexbufferInfo *vbInfo->eltResource->rmResource。vbInfo的resource是什么时候创建的?
      然后做blit,rmBufferBlt()里通过rmiCanBuf2dBlt()检查是否只能用cpublt,因为有POOL_SYSTEMMEM,所以是必须走CPUblt
      CPUblt里,先把src,dst的Data lock出来,然后memcpy(dst, src, copyBytes)
      对于usage=GL_STATIC_DRAW,driver里面不care。

    glEnableVertexAttribArray(0x0);

    • API: 入参index指generic vertex attribute index.
      AP通过它指定use either constant data or data from vertex array.
    • Driver: 在pVertexArrayState->arrayEnabled置位了相应的bit。
      这里我已经不明确什么是vertex arribute array了,要看看OpenGL编程指南。
      这里有个例子也许有帮助:
    //Example 6-3 Using Constant and Vertex Array Attributes
    int Init ( ESContext *esContext )
    {
        UserData *userData = (UserData*) esContext->userData;
        const char vShaderStr[] = 
            "#version 300 es \n"
            "layout(location = 0) in vec4 a_color; \n"
            "layout(location = 1) in vec4 a_position; \n"
            "out vec4 v_color; \n"
            "void main() \n"
            "{ v_color = a_color; \n"
            " gl_Position = a_position; }\n" ;
    
        const char fShaderStr[] = 
            "#version 300 es \n"
            "precision mediump float; \n"
            "in vec4 v_color; \n"
            "out vec4 o_fragColor; \n"
            "void main() \n"
            "{ o_fragColor = v_color; \n } " ;
    
            GLuint programObject;
            //Create program object;
            programObject = esLoadProgram(vShaderStr, fShaderStr);
            if(programObject == 0)  return GL_FALSE;
            //Store program object
            userData->programObject = programObject;
            glClearColor(0.0f, 0.0f, .0.0f, 0.0f );
            return GL_TRUE;
    }
    
    void Draw(ESCOntext *esContext)
    {
        UserData *userData = (UserData*)esContext->userData;
        GLfloat color[4] = {1.0f, 0.0f, 0.0f, 1.0f};
        //3 vertices, with(x,y,z)per-vertex
        GLfloat vertexPos[3*3] =
        { 0.0f,0.5f,0.0f, //v0
          -0.5f,-0.5f,0.0f, //v1
          0.5f,-0.5f,0.0f //v2
        };
    
        glViewport(0, 0, esContext->width, esContext->height);
        //
        glClear(GL_COLOR_BUFFER_BIT);
        glUseProgram(userData->programObject);
        glVertexAttrib4fb(0, color);
        glVettexAttribPointer(1, 3, GL_FLOAT, GL_FALSE, 0, vertexPos);
        glEnableVertexAttribArray(1);
        glDrawArrays(GL_TRANINGES, 0 ,3);
        glDisplayVertexAttribArray(1);
    }
    

    vertex attribute:color,用glVertexAttrib4fb指定一个常量,没有enable vertex attribute array 0。
    vertexPos这个attribute,通过glVertexAttribPointer指定了使用vertex array,并enable array(by glEnableVertexAttribArray)
    所以,color值对所有vertices of triangle都是一样的,但是每个vertices的vertexPos attribute是不同的。
    上面的code用到了glVertexAttrib4fv(index, *values);用于指定Vertex Attribute Data,
    Vertex attribute data,可以通过vertex array为每个vertex指定,也可以把一个constant value用于primitive的所有vertices。

    • Constant Vertex Attribute
      a constan vertex attribute is same for all vertices of a primitive

    • Vertex Arrays
      顶点数组指定每个vertex的attribute dataa,buffer保存在client space

    void glVertexAttribPointer(index, size, type, normalized, stride, *ptr)
    index : specifies generic vertex attibute index
    size: 通过index引用的vertex attirbute的分量,1-4
    stirde: as pitch(跨距) to get vertex data for next index.
    ptr : 使用client-size vertex array时,指向持有vertex attibute data的buffer; 使用vertex buffer object时,表示offset in that buffer。
    

    stirde: as pitch(跨距) to get vertex data for next index.
    ptr : 使用client-size vertex array时,指向持有vertex attibute data的buffer; 使用vertex buffer object时,表示offset in that buffer。

    先走着


    texture:

    glActiveTexture(GL_TEXTURE0);

    • API: 9.1.13 Using Textures in a Shader
      example : Vertex and Fragment Shaders for Perfroming 2D Texturing : Simple_Texture2D sample:
    //Vertex shader
    #version 300 es
    layout(location = 0) in vec4 a_position;
    layout(location = 1) in vec2 a_texCoord;
    out vec2 v_texCorrd;
    void main()
    {
        gl_Position = a_position;
         v_texCoord = a_texCoord;
    }
    
    //Fragment shader
    #version 300 es
    precision mediump float;
    in vec2 v_texCoord;
    layout(location = 0) out vec4 outColor;
    uniform sampler2D s_texture;
    void main()
    {
        outColor = texture( s_texture, v_texCoord );
    }
    

    vertex shader以一个two-component texture coordinate作为vertex input, then passes it as output to framgnet shader.
    fragment shader consumes the texture coordinate and used it for texture fetch(读取).
    fragment shader声明了一个类型为sampler2D的uniform variable,called s_texture。

    sampler是uniform变量的一个特殊类型,用来fetch from a texture map,读取纹理贴图。
    sampler uniform will be loaded with a value specifying texture unit to which texture is bound;
    比如,specifying a sampler with value of 0, says to fetch from unit GL_TEXTURE0,specifying a value of 1 says to fetch from GL_TEXTURE1, and so on.

    Textures are bound to texture units by glActiveTexture(GLenum texture)
    texture: texture unit to make active: GL_TEXTURE0/1/2/3/.../31/
    glActiveTexture设置了current texture unit,以便后续的glBindTexutre()会Bind texture to currently active uint。
    example show how sampler and texture are bound to texture unit:

    //get the sampler locations
    userData->samplerLoc = glGetUniformLocation(userData->programObject,"s_texture");
    
    //bind texture
    glActiveTexture(GL_TEXTURE0);
    glBindTexture(GL_TEXTURE_2D, userData->textureId);
    
    //set sampler texture unit to 0
    glUniformli(userData->samplerLoc, 0);
    

    到这里,我们已经load了texture, texturen bound to texture unit 0 , sampler set to use texture uint 0.
    回到simple_texture2D sample,可以看到,shader code使用了内建函数texture,来fetch from texture map,

    vec4 texture(sampler2D sampler, vec2 coord[,float bias])
    sampler: sampler bound to a texture unit, 来指定要fetch的texture
    coord: a 2D texture coordinate, 用来fetch from texture map
    bias: 可选参数,提供texture fetch所用的mipmap bias(mip贴图 偏置),
    

    texture()返回一个vec4,代表color fetched fromt texture map.
    texture data mapped到color channels的方式,依赖于texture base format。
    下表展示了texture format map到vec4 color的方式,texture swizzles决定了,每个分量的值,如何map到shader的分量

    base format Texel Data 纹素
    GL_RGB R G B 1.0
    GL_RGBA R G B A
    GL_LUMINANCE L L L 1.0
    GL_LUMINANCE_ALPHA L L L A
    GL_ALPHA 0.0 .0.0 0.0 A

    这里有个texture swizzles,9.1.9纹理调配讲了:texture swizzles control how color components in input R,RG,RGB,RGBA texture map to components when fetched from in the shader.
    感觉和allocation->flag.swizzled不一样,这个对应的是tiled,

    相关文章

      网友评论

          本文标题:Android VideoPlayer 调用的OpenGL ES

          本文链接:https://www.haomeiwen.com/subject/jtxjwctx.html