CoreVideo文档阅读及常见使用方式

作者: 西山薄凉 | 来源:发表于2020-03-10 00:10 被阅读0次

简述

平时工作中使用 CoreVideo 也不算少，但是一直没有系统完整地阅读梳理过它的官方文档。趁着这段时间较为闲暇，就系统性的学习一下官方文档，并且记录一些常见用法。

常见用法

复制 CVPixelBuffer

经常我们需要拷贝一个 CVPixelBuffer，但是没有直接拷贝的方法。这里就是一个缓冲拷贝的实现，并且对源和目标为不同大小的缓冲情况进行了处理。

bool copyPixelBuffer(CVPixelBufferRef src, CVPixelBufferRef dst) {
    bool ret = true;
    
    CVPixelBufferLockBaseAddress(src, kCVPixelBufferLock_ReadOnly);
    
    unsigned char* pb = (unsigned char*)CVPixelBufferGetBaseAddressOfPlane(src, 0);
    int height = (int)CVPixelBufferGetHeight(src);
    int stride = (int)CVPixelBufferGetBytesPerRow(src);
    int size = (int)CVPixelBufferGetDataSize(src);
    
    while (1) {
        CVReturn cvRet = CVPixelBufferLockBaseAddress(dst, 0);
        if (cvRet != kCVReturnSuccess) {
            ret = false;
            break;
        }
        
        int dst_height = (int)CVPixelBufferGetHeight(dst);
        int dst_stride = (int)CVPixelBufferGetBytesPerRow(dst);
        int dst_size = (int)CVPixelBufferGetDataSize(dst);
        
        if (stride == dst_stride && dst_size == size) {
            unsigned char* temp = (unsigned char*)CVPixelBufferGetBaseAddressOfPlane(dst, 0);
            memcpy(temp, pb, size);
        } else {
            int copy_height = height > dst_height ? dst_height : height;
            int copy_stride = stride > dst_stride ? dst_stride : stride;
            
            unsigned char* offset_dst = (unsigned char*)CVPixelBufferGetBaseAddressOfPlane(dst, 0);
            unsigned char* offset_src = pb;
            for (int i = 0; i < copy_height; i++) {
                memcpy(offset_dst, offset_src, copy_stride);
                offset_src += stride;
                offset_dst += dst_stride;
            }
        }
        
        CVPixelBufferUnlockBaseAddress(dst, 0);
        break;
    }
    
    CVPixelBufferUnlockBaseAddress(src, kCVPixelBufferLock_ReadOnly);
    return ret;
}

通过 CGImage 创建 CVPixelBuffer

CVPixelBufferRef pixelBufferFromCGImage(CGImageRef image) {
    CVPixelBufferRef pxbuffer = NULL;
    NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
                             [NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey,
                             [NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey,
                             nil];
    
    size_t width =  CGImageGetWidth(image);
    size_t height = CGImageGetHeight(image);
    size_t bytesPerRow = CGImageGetBytesPerRow(image);
    
    CFDataRef  dataFromImageDataProvider = CGDataProviderCopyData(CGImageGetDataProvider(image));
    GLubyte  *imageData = (GLubyte *)CFDataGetBytePtr(dataFromImageDataProvider);
    
    CVPixelBufferCreateWithBytes(kCFAllocatorDefault,width,height,kCVPixelFormatType_32BGRA,imageData,bytesPerRow,NULL,NULL,(__bridge CFDictionaryRef)options,&pxbuffer);
    
    CFRelease(dataFromImageDataProvider);
    
    return pxbuffer;
}

CVPixelBuffer 转 CGImage

CGImageRef createCGImageFromCVPixelBuffer(CVPixelBufferRef pixels) {
    
    CVPixelBufferLockBaseAddress(pixels, kCVPixelBufferLock_ReadOnly);
    
    CIImage *ciImage = [CIImage imageWithCVPixelBuffer:pixels];
    CIContext *temporaryContext = [CIContext contextWithOptions:nil];
    CGImageRef videoImage = [temporaryContext createCGImage:ciImage fromRect:CGRectMake(0, 0, CVPixelBufferGetWidth(pixels), CVPixelBufferGetHeight(pixels))];
    
    CVPixelBufferUnlockBaseAddress(pixels, kCVPixelBufferLock_ReadOnly);
    
    return videoImage;
}

创建BGRA/I420/NV12 格式的 CVPixelBufferPool

+ (bool)create32BGRAPixelBufferPool:(CVPixelBufferPoolRef*)pool width:(int)width height:(int)height {
    CFDictionaryRef empty; // empty value for attr value.
    CFMutableDictionaryRef attrs;
    
    empty = CFDictionaryCreate(kCFAllocatorDefault,
                               NULL, NULL, 0,
                               &kCFTypeDictionaryKeyCallBacks,
                               &kCFTypeDictionaryValueCallBacks); // our empty IOSurface properties dictionary
    
    SInt32 cvPixelFormatTypeValue = kCVPixelFormatType_32BGRA;
    CFNumberRef cfPixelFormat = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, (const void*)(&(cvPixelFormatTypeValue)));
    
    SInt32 cvWidthValue = width;
    CFNumberRef cfWidth = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, (const void*)(&(cvWidthValue)));
    SInt32 cvHeightValue = height;
    CFNumberRef cfHeight = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, (const void*)(&(cvHeightValue)));
    
    attrs = CFDictionaryCreateMutable(kCFAllocatorDefault,
                                      4,
                                      &kCFTypeDictionaryKeyCallBacks,
                                      &kCFTypeDictionaryValueCallBacks);
    
    CFDictionarySetValue(attrs, kCVPixelBufferIOSurfacePropertiesKey, empty);
    CFDictionarySetValue(attrs, kCVPixelBufferPixelFormatTypeKey, cfPixelFormat);
    CFDictionarySetValue(attrs, kCVPixelBufferWidthKey, cfWidth);
    CFDictionarySetValue(attrs, kCVPixelBufferHeightKey, cfHeight);
    
    CVReturn ret = CVPixelBufferPoolCreate(kCFAllocatorDefault, nil, attrs, pool);
    
    CFRelease(attrs);
    CFRelease(empty);
    CFRelease(cfPixelFormat);
    CFRelease(cfWidth);
    CFRelease(cfHeight);
    
    if (ret != kCVReturnSuccess) {
        return false;
    }
    
    return true;
}

+ (bool)createI420PixelBufferPool:(CVPixelBufferPoolRef*)pool width:(int)width height:(int)height {
    CFDictionaryRef empty; // empty value for attr value.
    CFMutableDictionaryRef attrs;
    
    empty = CFDictionaryCreate(kCFAllocatorDefault,
                               NULL, NULL, 0,
                               &kCFTypeDictionaryKeyCallBacks,
                               &kCFTypeDictionaryValueCallBacks); // our empty IOSurface properties dictionary
    
    SInt32 cvPixelFormatTypeValue = kCVPixelFormatType_420YpCbCr8PlanarFullRange;
    CFNumberRef cfPixelFormat = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, (const void*)(&(cvPixelFormatTypeValue)));
    
    SInt32 cvWidthValue = width;
    CFNumberRef cfWidth = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, (const void*)(&(cvWidthValue)));
    SInt32 cvHeightValue = height;
    CFNumberRef cfHeight = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, (const void*)(&(cvHeightValue)));
    
    attrs = CFDictionaryCreateMutable(kCFAllocatorDefault,
                                      5,
                                      &kCFTypeDictionaryKeyCallBacks,
                                      &kCFTypeDictionaryValueCallBacks);
    
    CFDictionarySetValue(attrs, kCVPixelBufferIOSurfacePropertiesKey, empty);
    CFDictionarySetValue(attrs, kCVPixelBufferPixelFormatTypeKey, cfPixelFormat);
    CFDictionarySetValue(attrs, kCVPixelBufferWidthKey, cfWidth);
    CFDictionarySetValue(attrs, kCVPixelBufferHeightKey, cfHeight);
#if TARGET_OS_IOS
    CFDictionarySetValue(attrs, kCVPixelBufferOpenGLESCompatibilityKey, kCFBooleanTrue);
#elif TARGET_OS_OSX
    CFDictionarySetValue(attrs, kCVPixelBufferOpenGLCompatibilityKey, kCFBooleanTrue);
#endif
    
    CVReturn ret = CVPixelBufferPoolCreate(kCFAllocatorDefault, nil, attrs, pool);
    
    CFRelease(attrs);
    CFRelease(empty);
    CFRelease(cfPixelFormat);
    CFRelease(cfWidth);
    CFRelease(cfHeight);
    
    if (ret != kCVReturnSuccess) {
        return false;
    }
    
    return true;
}

+ (bool)createNV12PixelBufferPool:(CVPixelBufferPoolRef*)pool width:(int)width height:(int)height {
    CFDictionaryRef empty; // empty value for attr value.
    CFMutableDictionaryRef attrs;
    
    empty = CFDictionaryCreate(kCFAllocatorDefault,
                               NULL, NULL, 0,
                               &kCFTypeDictionaryKeyCallBacks,
                               &kCFTypeDictionaryValueCallBacks); // our empty IOSurface properties dictionary
    
    SInt32 cvPixelFormatTypeValue = kCVPixelFormatType_420YpCbCr8BiPlanarFullRange;
    CFNumberRef cfPixelFormat = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, (const void*)(&(cvPixelFormatTypeValue)));
    
    SInt32 cvWidthValue = width;
    CFNumberRef cfWidth = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, (const void*)(&(cvWidthValue)));
    SInt32 cvHeightValue = height;
    CFNumberRef cfHeight = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, (const void*)(&(cvHeightValue)));
    
    attrs = CFDictionaryCreateMutable(kCFAllocatorDefault,
                                      5,
                                      &kCFTypeDictionaryKeyCallBacks,
                                      &kCFTypeDictionaryValueCallBacks);
    
    CFDictionarySetValue(attrs, kCVPixelBufferIOSurfacePropertiesKey, empty);
    CFDictionarySetValue(attrs, kCVPixelBufferPixelFormatTypeKey, cfPixelFormat);
    CFDictionarySetValue(attrs, kCVPixelBufferWidthKey, cfWidth);
    CFDictionarySetValue(attrs, kCVPixelBufferHeightKey, cfHeight);
#if TARGET_OS_IOS
    CFDictionarySetValue(attrs, kCVPixelBufferOpenGLESCompatibilityKey, kCFBooleanTrue);
#elif TARGET_OS_OSX
    CFDictionarySetValue(attrs, kCVPixelBufferOpenGLCompatibilityKey, kCFBooleanTrue);
#endif
    
    CVReturn ret = CVPixelBufferPoolCreate(kCFAllocatorDefault, nil, attrs, pool);
    
    CFRelease(attrs);
    CFRelease(empty);
    CFRelease(cfPixelFormat);
    CFRelease(cfWidth);
    CFRelease(cfHeight);
    
    if (ret != kCVReturnSuccess) {
        return false;
    }
    
    return true;
}

官方文档

DataProcessing

CVBuffer

CV 缓冲基类。派生了 CVImageBuffer。

Attachment

You can attach any Core Foundation object to a Core Video buffer to store additional information.
您可以将任何Core Foundation对象附加到 CVBuffer 以存储其他信息。
提供附加信息的增删改查及复制方法。

Retain Count

修改 CVBuffer 的引用计数值。支持传空。

CVImageBuffer

提供管理不同类型图像的接口。

Inspect

查看 Buffer 参数，如颜色空间、显示大小、编码尺寸、翻转状态等。

Create Color Speaces

由附件创建色彩空间。
色彩空间由附件字典中信息生成。

Data Type

CVImageBufferRef

Converting Between Strings and Integer Code Points

看上去是新添加上去的，转换相关的方法。

CVPixelBuffer

表示保存在主存的像素。

Create

支持以各种方式创建 PixelBuffer。
如直接初始化一块空的像素缓冲区，或者直接引用现有的一整段内存将其初始化为缓冲，又或者通过内存中的几个平面数据初始化缓冲，或者使用IOSurface（跨进程共享）初始化一个缓冲。
那么需要注意的是，后几种方式由于是引用了一段内存，所以缓冲释放不会释放引用的内存，而使通过回调通知开发者处理缓冲释放事件。

Inspect

查看缓冲参数，如基址、平面基址、宽高、每行字节数，判断缓冲类型、平面数、平面大小等。

Modify

主要是锁定基址和解锁基址两个方法。
需要在 CPU 访问缓存前后分别调用。如果使用 GPU 访问，则不需要锁定，并且锁定会影响性能。

Retain Count

同 CVBufferRetain

CVPixelBufferPool

缓冲池，用于优化内存分配的性能。

Create

创建缓冲池，通过缓冲池创建 PixelBuffer。
并且可以指定创建出的像素缓冲默认携带指定的 Attributes。

Flush

释放所有未使用的缓冲区。

Inspect

获取缓冲池属性字典、缓冲属性字典。

Retain Count

同上

CVPixelFormatDescription

像素格式说明。在需要自定义像素格式时，才使用该说明对象。

Create

创建格式说明。

Retrieve

检索已知的定义的所有像素格式描述。

Time Management

Inspect Host Clock

检查主机时钟，包括当前系统时间、系统时间更新频率、系统时间最小增量。

CVTime

用来表示时间的数据结构。

typedef struct
{
    int64_t     timeValue;//时间分母
    int32_t     timeScale;//时间分子
    int32_t     flags;//可以表示起始时间，或者未知时间、无限时间
} CVTime;

CVTimeStamp

用于定义显示时间戳的数据结构。

CVDisplayLink

类似 CADisplayLink。
是一个与屏幕刷新同步调用的 Timer。

CVMetalTextureCache

Create

创建新纹理缓存、由现有图片生成纹理缓存、回收/整理当前缓存。

CoreVideo文档阅读及常见使用方式

简述

常见用法

复制 CVPixelBuffer

通过 CGImage 创建 CVPixelBuffer

CVPixelBuffer 转 CGImage

创建BGRA/I420/NV12 格式的 CVPixelBufferPool

官方文档

DataProcessing

CVBuffer

Attachment

Retain Count

CVImageBuffer

Inspect

Create Color Speaces

Data Type

Converting Between Strings and Integer Code Points

CVPixelBuffer

Create

Inspect

Modify

Retain Count

CVPixelBufferPool

Create

Flush

Inspect

Retain Count

CVPixelFormatDescription

Create

Retrieve

Time Management

Inspect Host Clock

CVTime

CVTimeStamp

CVDisplayLink

CVMetalTextureCache

Create

CVMetalTexture

Inspect

OpenGL/OpenGL ES

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读