簡單瞭解 iOS CVPixelBuffer （上）
theme: smartblue
「這是我參與2022首次更文挑戰的第8天，活動詳情檢視：2022首次更文挑戰」。
前言：
在iOS中，我們會常常看到 CVPixelBufferRef 這個型別，最常見到的場景是在Camera 採集的時候，返回的資料中有一個CMSampleBufferRef，而每個CMSampleBufferRef則包含一個 CVPixelBufferRef，在視訊硬解碼的返回資料裡也是一個 CVPixelBufferRef（裡面包含了所有的壓縮的圖片資訊）。在瞭解了CVPixelBufferRef之後，我們將能夠掌握並且運用CVPixelBufferRef的使用；
本篇我們主要熟悉下CVPixelBuffer的使用；
CVPixelBuffer 簡介
CVPixelBuffer：核心視訊畫素緩衝區是在主儲存器中儲存畫素的影象緩衝區。生成幀、壓縮或解壓縮視訊或使用 Core Image 的應用程式都可以使用 CVPixelBuffer。
CVPixelBufferRef：是畫素緩衝區型別，對CVPixelBuffer物件的引用。畫素緩衝區型別基於影象緩衝區型別；畫素緩衝器實現了影象緩衝器的儲存器儲存。
```
/*!
@typedef CVPixelBufferRef
@abstract Based on the image buffer type. The pixel buffer implements the memory storage for an image buffer.
*/ typedef CVImageBufferRef CVPixelBufferRef; ``` 由於CVPixelBufferRef是C中的物件，所以沒有ARC記憶體管理，必須由開發者自己去管理引用計數，控制物件生命週期；
首先，我們先來了解有關CVPixelBufferRef的基礎方法，例如：CVPixelBufferRef的建立、持有、釋放；
建立
建立的方法有四種，前三種常用到： - CVPixelBufferCreate() - CVPixelBufferCreateWithBytes() - CVPixelBufferCreateWithPlanarBytes() - CVPixelBufferCreateWithIOSurface()
普通的建立方法
/*!
為給定大小和畫素格式建立單個畫素緩衝區
@function CVPixelBufferCreate
@abstract Call to create a single PixelBuffer for a given size and pixelFormatType.
@discussion Creates a single PixelBuffer for a given size and pixelFormatType. It allocates the necessary memory based on the pixel dimensions, pixelFormatType and extended pixels described in the pixelBufferAttributes. Not all parameters of the pixelBufferAttributes will be used here.
@param width Width of the PixelBuffer in pixels.
@param height Height of the PixelBuffer in pixels.
@param pixelFormatType Pixel format indentified by its respective OSType.
@param pixelBufferAttributes A dictionary with additional attributes for a pixel buffer. This parameter is optional. See BufferAttributeKeys for more details.
@param pixelBufferOut The new pixel buffer will be returned here
@result returns kCVReturnSuccess on success.
*/
CV_EXPORT CVReturn CVPixelBufferCreate(
CFAllocatorRef CV_NULLABLE allocator,
size_t width,
size_t height,
OSType pixelFormatType,
CFDictionaryRef CV_NULLABLE pixelBufferAttributes,
CV_RETURNS_RETAINED_PARAMETER CVPixelBufferRef CV_NULLABLE * CV_NONNULL pixelBufferOut) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0);
建立一個由記憶體位置指定資料的畫素緩衝區
``` /*! 為給定大小和畫素格式建立一個畫素緩衝區，其中包含由記憶體位置指定的資料 @function CVPixelBufferCreateWithBytes @abstract Call to create a single PixelBuffer for a given size and pixelFormatType based on a passed in piece of memory. @discussion Creates a single PixelBuffer for a given size and pixelFormatType. Not all parameters of the pixelBufferAttributes will be used here. It requires a release callback function that will be called, when the PixelBuffer gets destroyed so that the owner of the pixels can free the memory. @param width Width of the PixelBuffer in pixels @param height Height of the PixelBuffer in pixels @param pixelFormatType Pixel format indentified by its respective OSType. @param baseAddress Address of the memory storing the pixels. @param bytesPerRow Row bytes of the pixel storage memory. @param releaseCallback CVPixelBufferReleaseBytePointerCallback function that gets called when the PixelBuffer gets destroyed. @param releaseRefCon User data identifying the PixelBuffer for the release callback. @param pixelBufferAttributes A dictionary with additional attributes for a a pixel buffer. This parameter is optional. See PixelBufferAttributes for more details. @param pixelBufferOut The new pixel buffer will be returned here @result returns kCVReturnSuccess on success.
*/
CV_EXPORT CVReturn CVPixelBufferCreateWithBytes( CFAllocatorRef CV_NULLABLE allocator, size_t width, size_t height, OSType pixelFormatType, void * CV_NONNULL baseAddress, size_t bytesPerRow, CVPixelBufferReleaseBytesCallback CV_NULLABLE releaseCallback, void * CV_NULLABLE releaseRefCon, CFDictionaryRef CV_NULLABLE pixelBufferAttributes, CV_RETURNS_RETAINED_PARAMETER CVPixelBufferRef CV_NULLABLE * CV_NONNULL pixelBufferOut) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0); ```
建立一個平面格式的CVPixelBuffer
/*!
@function CVPixelBufferCreateWithPlanarBytes
@abstract Call to create a single PixelBuffer in planar format for a given size and pixelFormatType based on a passed in piece of memory.
@discussion Creates a single PixelBuffer for a given size and pixelFormatType. Not all parameters of the pixelBufferAttributes will be used here. It requires a release callback function that will be called, when the PixelBuffer gets destroyed so that the owner of the pixels can free the memory.
@param width Width of the PixelBuffer in pixels
@param height Height of the PixelBuffer in pixels
@param pixelFormatType Pixel format indentified by its respective OSType.
@param dataPtr Pass a pointer to a plane descriptor block, or NULL.
@param dataSize pass size if planes are contiguous, NULL if not.
@param numberOfPlanes Number of planes.
@param planeBaseAddress Array of base addresses for the planes.
@param planeWidth Array of plane widths.
@param planeHeight Array of plane heights.
@param planeBytesPerRow Array of plane bytesPerRow values.
@param releaseCallback CVPixelBufferReleaseBytePointerCallback function that gets called when the PixelBuffer gets destroyed.
@param releaseRefCon User data identifying the PixelBuffer for the release callback.
@param pixelBufferAttributes A dictionary with additional attributes for a a pixel buffer. This parameter is optional. See PixelBufferAttributes for more details.
@param pixelBufferOut The new pixel buffer will be returned here
@result returns kCVReturnSuccess on success.
*/
CV_EXPORT CVReturn CVPixelBufferCreateWithPlanarBytes(
CFAllocatorRef CV_NULLABLE allocator,
size_t width,
size_t height,
OSType pixelFormatType,
void * CV_NULLABLE dataPtr, // pass a pointer to a plane descriptor block, or NULL
size_t dataSize, // pass size if planes are contiguous, NULL if not
size_t numberOfPlanes,
void * CV_NULLABLE planeBaseAddress[CV_NONNULL ],
size_t planeWidth[CV_NONNULL ],
size_t planeHeight[CV_NONNULL ],
size_t planeBytesPerRow[CV_NONNULL ],
CVPixelBufferReleasePlanarBytesCallback CV_NULLABLE releaseCallback,
void * CV_NULLABLE releaseRefCon,
CFDictionaryRef CV_NULLABLE pixelBufferAttributes,
CV_RETURNS_RETAINED_PARAMETER CVPixelBufferRef CV_NULLABLE * CV_NONNULL pixelBufferOut) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0);
修改
在使用 CPU 訪問畫素資料之前必須呼叫CVPixelBufferLockBaseAddress,然後再呼叫CVPixelBufferUnlockBaseAddress。
而使用 GPU 訪問畫素資料時，就沒有這個必要了。
填充擴充套件的PixelBuffer
``` /*! 填充擴充套件的PixelBuffer @function CVPixelBufferFillExtendedPixels @abstract Fills the extended pixels of the PixelBuffer. This function replicates edge pixels to fill the entire extended region of the image. @param pixelBuffer Target PixelBuffer.
*/
CV_EXPORT CVReturn CVPixelBufferFillExtendedPixels( CVPixelBufferRef CV_NONNULL pixelBuffer ) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0); ```
鎖定PixelBuffer地址
``` /*! 鎖定PixelBuffer地址 @function CVPixelBufferLockBaseAddress @abstract Description Locks the BaseAddress of the PixelBuffer to ensure that the memory is accessible. @discussion This API ensures that the CVPixelBuffer is accessible in system memory. This should only be called if the base address is going to be used and the pixel data will be accessed by the CPU. @param pixelBuffer Target PixelBuffer. @param lockFlags See CVPixelBufferLockFlags. @result kCVReturnSuccess if the lock succeeded, or error code on failure
*/
CV_EXPORT CVReturn CVPixelBufferLockBaseAddress( CVPixelBufferRef CV_NONNULL pixelBuffer, CVPixelBufferLockFlags lockFlags ) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0);
```
解鎖PixelBuffer地址
``` /*!
@function CVPixelBufferUnlockBaseAddress
@abstract Description Unlocks the BaseAddress of the PixelBuffer.
@param pixelBuffer Target PixelBuffer.
@param unlockFlags See CVPixelBufferLockFlags.
@result kCVReturnSuccess if the unlock succeeded, or error code on failure
*/
CV_EXPORT CVReturn CVPixelBufferUnlockBaseAddress( CVPixelBufferRef CV_NONNULL pixelBuffer, CVPixelBufferLockFlags unlockFlags ) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0); ```
記憶體管理（保留或者釋放）
這裡在開篇的時候也提到過，由於CVPixelBufferRef是C中的物件，所以不存在ARC記憶體管理，由開發者自己去管理引用計數，從而控制物件的生命週期；
持有
/*!
引用計數+1
@function CVPixelBufferRetain
@abstract Retains a CVPixelBuffer object
@discussion Equivalent to CFRetain, but NULL safe
@param buffer A CVPixelBuffer object that you want to retain.
@result A CVPixelBuffer object that is the same as the passed in buffer.
*/
CV_EXPORT CVPixelBufferRef CV_NULLABLE CVPixelBufferRetain( CVPixelBufferRef CV_NULLABLE texture ) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0);
釋放
/*!
引用計數-1
@function CVPixelBufferRelease
@abstract Releases a CVPixelBuffer object
@discussion Equivalent to CFRelease, but NULL safe
@param buffer A CVPixelBuffer object that you want to release.
*/
CV_EXPORT void CVPixelBufferRelease( CV_RELEASES_ARGUMENT CVPixelBufferRef CV_NULLABLE texture ) __OSX_AVAILABLE_STARTING(__MAC_10_4,__IPHONE_4_0);
檢查
CVPixelBufferGetHeight 返回畫素緩衝區的高度
CVPixelBufferGetWidth 返回畫素緩衝區的寬度
這裡方法較多，就不一一舉例，可以檢視官方文件
畫素格式型別 kCVPixelFormatType
RGB :
kCVPixelFormatType_32BGRA = 'BGRA'
kCVPixelFormatType_32BGRA = 'BGRA',
kCVPixelFormatType_32ABGR = 'ABGR',
kCVPixelFormatType_32RGBA = 'RGBA',
NV12 :
kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange = '420v',
kCVPixelFormatType_420YpCbCr8BiPlanarFullRange = '420f',
YUV420P :
kCVPixelFormatType_420YpCbCr8Planar = 'y420',
kCVPixelFormatType_{長度|序列}{顏色空間}{Planar|BiPlanar}{VideoRange|FullRange}
從這裡型別可以看出，YUV格式和RGB格式都是可以建立成CVPixelBuffer；
那麼拿到CVPixelBufferRef後，如果要顯示的話， 我們可以通過CVPixelBufferRef轉成UIImage，或者通過繪製紋理的形式展示。
想要更清楚的瞭解kCVPixelFormatType這些型別，那麼就需要我們對YUV格式和RGB格式有更深入的瞭解，所以在下一篇中，我們將對兩種格式進行詳細的說明，然後再來看kCVPixelFormatType，那樣將會加深我們對此的瞭解程度。
- LeetCode 初級演算法之陣列（上），看看你都學會了嗎？
- LeetCode 初級演算法之連結串列，看看你都學會了嗎？
- LeetCode 初級演算法之字串（上），看看你都學會了嗎？
- 純程式碼佈局，也可以一樣的簡潔
- UIStackView之一問一答
- 使用UIStackView來簡化iOS的介面佈局
- 夏天來了，iOS開發者們該如何減少App耗電？（上）
- 夏天來了，App開發者們如何看待手機發燙問題？
- 聊聊iOS中UITableView複用的那些事
- 曾經經典的微信打飛機遊戲還有人記得嗎？
- iOS 原生渲染與 Flutter 有什麼區別 （上）
- 瞭解 Mach-O檔案
- CocoaPods中podsepc檔案設定詳解
- iOS 原生渲染與 Flutter 有什麼區別 （下）
- 簡單瞭解 iOS CVPixelBuffer （上）
- 談談 iOS 包瘦身方案
- 播放器重構的探索之路
- 如何使用CocoaPods製作私有庫
- iOS 元件化方案