从Swift中的CVPixelBufferRef获取像素值

Question

如何从CVPixelBufferRef获取RGB(或任何其他格式)像素值？我尝试了很多方法,但还没有成功.

func captureOutput(captureOutput: AVCaptureOutput!,
                   didOutputSampleBuffer sampleBuffer: CMSampleBuffer!,
                   fromConnection connection: AVCaptureConnection!) {
  let pixelBuffer: CVPixelBufferRef = CMSampleBufferGetImageBuffer(sampleBuffer)!
                CVPixelBufferLockBaseAddress(pixelBuffer, 0)
  let baseAddress = CVPixelBufferGetBaseAddress(pixelBuffer)

  //Get individual pixel values here

  CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)
}

Answer 1

baseAddress是一个不安全的可变指针或更确切地说是一个UnsafeMutablePointer<Void>.将指针转换Void为更具体的类型后,可以轻松访问内存:

// Convert the base address to a safe pointer of the appropriate type
let byteBuffer = UnsafeMutablePointer<UInt8>(baseAddress)

// read the data (returns value of type UInt8)
let firstByte = byteBuffer[0]

// write data
byteBuffer[3] = 90

确保使用正确的类型(8,16或32位无符号整数).这取决于视频格式.最有可能是8位.

更新缓冲区格式:

您可以在初始化AVCaptureVideoDataOutput实例时指定格式.你基本上可以选择:

• BGRA:单个平面,其中蓝色,绿色,红色和alpha值分别以32位整数存储
• 420YpCbCr8BiPlanarFullRange:两个平面,第一个包含每个像素的字节,具有Y(亮度)值,第二个包含像素组的Cb和Cr(色度)值
• 420YpCbCr8BiPlanarVideoRange:与420YpCbCr8BiPlanarFullRange相同,但Y值限制在16 - 235范围内(由于历史原因)

如果您对颜色值感兴趣并且速度(或者说最大帧速率)不是问题,那么请选择更简单的BGRA格式.否则采取一种更有效的原生视频格式.

如果您有两个平面,则必须获取所需平面的基址(请参阅视频格式示例):

视频格式示例

let pixelBuffer: CVPixelBufferRef = CMSampleBufferGetImageBuffer(sampleBuffer)!
CVPixelBufferLockBaseAddress(pixelBuffer, 0)
let baseAddress = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 0)
let bytesPerRow = CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 0)
let byteBuffer = UnsafeMutablePointer<UInt8>(baseAddress)

// Get luma value for pixel (43, 17)
let luma = byteBuffer[17 * bytesPerRow + 43]

CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)

BGRA的例子

let pixelBuffer: CVPixelBufferRef = CMSampleBufferGetImageBuffer(sampleBuffer)!
CVPixelBufferLockBaseAddress(pixelBuffer, 0)
let baseAddress = CVPixelBufferGetBaseAddress(pixelBuffer)
let int32PerRow = CVPixelBufferGetBytesPerRow(pixelBuffer)
let int32Buffer = UnsafeMutablePointer<UInt32>(baseAddress)

// Get BGRA value for pixel (43, 17)
let luma = int32Buffer[17 * int32PerRow + 43]

CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)

Answer 2

这是从 BGRA 像素缓冲区获取各个 rgb 值的方法。注意：您的缓冲区必须在调用之前锁定。

func pixelFrom(x: Int, y: Int, movieFrame: CVPixelBuffer) -> (UInt8, UInt8, UInt8) {
    let baseAddress = CVPixelBufferGetBaseAddress(movieFrame)
    
    let bytesPerRow = CVPixelBufferGetBytesPerRow(movieFrame)
    let buffer = baseAddress!.assumingMemoryBound(to: UInt8.self)
    
    let index = x*4 + y*bytesPerRow
    let b = buffer[index]
    let g = buffer[index+1]
    let r = buffer[index+2]
    
    return (r, g, b)
}

Answer 3

Swift3的更新:

let pixelBuffer: CVPixelBufferRef = CMSampleBufferGetImageBuffer(sampleBuffer)!
CVPixelBufferLockBaseAddress(pixelBuffer, CVPixelBufferLockFlags(rawValue: 0));
let int32Buffer = unsafeBitCast(CVPixelBufferGetBaseAddress(pixelBuffer), to: UnsafeMutablePointer<UInt32>.self)
let int32PerRow = CVPixelBufferGetBytesPerRow(pixelBuffer)
// Get BGRA value for pixel (43, 17)
let luma = int32Buffer[17 * int32PerRow + 43]

CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)