标签: cmsamplebuffer

我正在使用AVFoundation captureOutput didOutputSampleBuffer提取图像，然后用于过滤器。

  self.bufferFrameQueue = DispatchQueue(label: "bufferFrame queue", qos: DispatchQoS.background,  attributes: [], autoreleaseFrequency: .inherit)\n\n  self.videoDataOutput = AVCaptureVideoDataOutput()\n  if self.session.canAddOutput(self.videoDataOutput) {\n       self.session.addOutput(videoDataOutput)\n       self.videoDataOutput!.alwaysDiscardsLateVideoFrames = true\n       self.videoDataOutput!.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String: Int(kCVPixelFormatType_32BGRA)]\n           self.videoDataOutput!.setSampleBufferDelegate(self, queue: self.bufferFrameQueue)\n  }\n\n\n\n func captureOutput(_ captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, from connection: AVCaptureConnection!) {\n\n        connection.videoOrientation = .portrait\n\n        let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!\n        let ciImage = CIImage(cvPixelBuffer: pixelBuffer)\n\n        DispatchQueue.main.async {\n            self.cameraBufferImage = ciImage\n        }\n}\n

上面只是更新了self.cameraBufferImage上面只是在有新的输出样本缓冲区时

然后，当按下滤镜按钮时，我使用self.cameraBufferImage如下：

  func filterButtonPressed() {\n\n    if var …

我正在开发一个基于 Swift 的 macOS 应用程序，我需要在其中捕获视频输入，但不将其显示在屏幕上......而不是显示视频，我想将缓冲数据发送到其他地方进行处理，并最终显示它在SceneKit场景中的对象上。

我有一个CameraInput有prepareCamera方法的类：

    fileprivate func prepareCamera() {
        self.videoSession = AVCaptureSession()
        self.videoSession.sessionPreset = AVCaptureSession.Preset.photo

        if let devices = AVCaptureDevice.devices() as? [AVCaptureDevice] {
            for device in devices {
                if device.hasMediaType(AVMediaType.video) {
                    cameraDevice = device

                    if cameraDevice != nil  {
                        do {
                            let input = try AVCaptureDeviceInput(device: cameraDevice)


                            if videoSession.canAddInput(input) {
                                videoSession.addInput(input)
                            }


                           } catch {
                            print(error.localizedDescription)
                        }
                    }
                }
            }

            let videoOutput = AVCaptureVideoDataOutput()
            videoOutput.setSampleBufferDelegate(self as AVCaptureVideoDataOutputSampleBufferDelegate, queue: DispatchQueue(label: "sample buffer delegate", attributes: …

我目前正在尝试通过首先将其转换为 CVPixelBuffer 然后使用 vImageRotate90_ARGB8888 来转换缓冲区来更改 CMSampleBuffer 的方向。我的代码的问题是当 vImageRotate90_ARGB8888 执行时，它立即崩溃。我知道有答案（比如这个或这个），但是所有这些解决方案在我的情况下都不起作用，我真的找不到任何类型的错误，或者想不出任何会导致这种行为的东西。我当前的代码如下：

- (CVPixelBufferRef)rotateBuffer:(CMSampleBufferRef)sampleBuffer {
CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
CVPixelBufferLockBaseAddress(pixelBuffer, 0);

size_t bytesPerRow = CVPixelBufferGetBytesPerRow(pixelBuffer);
size_t width = CVPixelBufferGetWidth(pixelBuffer);
size_t height = CVPixelBufferGetHeight(pixelBuffer);
size_t currSize = bytesPerRow * height * sizeof(unsigned char);
size_t bytesPerRowOut = 4 * height * sizeof(unsigned char);

OSType pixelFormat = CVPixelBufferGetPixelFormatType(pixelBuffer);

void *baseAddress = CVPixelBufferGetBaseAddress(pixelBuffer);
unsigned char *outPixelData = (unsigned char *)malloc(currSize);

vImage_Buffer sourceBuffer = {baseAddress, height, width, bytesPerRow};
vImage_Buffer destinationBuffer = {outPixelData, width, height, …

我需要转换CMSampleBuffer为Data格式.我正在使用一个第三方框架进行音频相关任务.该框架为我提供了对象中的流式传输(即实时音频)音频CMSampleBuffer.

像这样:

func didAudioStreaming(audioSample: CMSampleBuffer!) {
    //Here I need to conver this to Data format. 
    //Because I am using GRPC framework for Audio Recognization, 
}

请提供我的步骤来转换CMSampleBuffer到Data.

FYI

    let formatDesc:CMFormatDescription? = CMSampleBufferGetFormatDescription(audioSample)

    <CMAudioFormatDescription 0x17010d890 [0x1b453ebb8]> {
    mediaType:'soun' 
    mediaSubType:'lpcm' 
    mediaSpecific: {
        ASBD: {
            mSampleRate: 16000.000000 
            mFormatID: 'lpcm' 
            mFormatFlags: 0xc 
            mBytesPerPacket: 2 
            mFramesPerPacket: 1 
            mBytesPerFrame: 2 
            mChannelsPerFrame: 1 
            mBitsPerChannel: 16     } 
        cookie: {(null)} 
        ACL: {(null)}
        FormatList Array: {(null)} 
    } 
    extensions: {(null)}
}

我正在尝试在使用 AVFoundation 的 AVCaptureVideoDataOutput 录制的视频上添加水印/徽标。我遇到的问题是，一旦写入视频，UIImage 的透明部分就会变成黑色。我做错了什么？

CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(buffer);

....

....

CIImage *image = [[CIImage alloc] initWithData:logoData]; 
CVPixelBufferLockBaseAddress( pixelBuffer, 0 );

CIContext *ciContext = [CIContext contextWithOptions:nil];

CGColorSpaceRef cSpace = CGColorSpaceCreateDeviceRGB();

[ciContext render:image toCVPixelBuffer:pixelBuffer bounds:CGRectMake(image.extent.origin.x, image.extent.origin.y - 2, image.extent.size.width, image.extent.size.height) colorSpace:cSpace];


CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
CGColorSpaceRelease(cSpace);

此代码会导致内存泄漏和应用程序崩溃：

    var outputSamples = [Float]()

    assetReader.startReading()
    while assetReader.status == .reading {
        let trackOutput = assetReader.outputs.first!

        if let sampleBuffer = trackOutput.copyNextSampleBuffer(),
            let blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer) {
            let blockBufferLength = CMBlockBufferGetDataLength(blockBuffer)
            let sampleLength = CMSampleBufferGetNumSamples(sampleBuffer) * channelCount(from: assetReader)
            var data = Data(capacity: blockBufferLength)
            data.withUnsafeMutableBytes { (blockSamples: UnsafeMutablePointer<Int16>) in
                CMBlockBufferCopyDataBytes(blockBuffer, atOffset: 0, dataLength: blockBufferLength, destination: blockSamples)
                CMSampleBufferInvalidate(sampleBuffer)

                let processedSamples = process(blockSamples,
                                               ofLength: sampleLength,
                                               from: assetReader,
                                               downsampledTo: targetSampleCount)
                outputSamples += processedSamples
            }
        }
    }
    var paddedSamples = [Float](repeating: silenceDbThreshold, count: targetSampleCount)
    paddedSamples.replaceSubrange(0..<min(targetSampleCount, outputSamples.count), with: …

“CMSampleBufferGetImageBuffer”已被属性“CMSampleBuffer.imageBuffer”替换

CMSampleBufferGet.ImageBuffer 不起作用:) 似乎 Swift 4.2 的参数也发生了变化。

guard let pixelBuffer: CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }

整个功能。万一 ...

    func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
//        print("Camera was able to capture a frame:", Date())

        guard let pixelBuffer: CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }
        guard let model = try? VNCoreMLModel(for: ARS().model) else { return }
        let request = VNCoreMLRequest(model: model) { (finishedReq, err) in

            guard let results = finishedReq.results as? [VNClassificationObservation] else { return } …

我目前正在开发一个在Swift中使用实时摄像头视图的项目.我使用了一些我在GitHub上找到的代码给我实时摄像机视图,它在运行Mojave的MacBook Pro上运行良好.我将所有文件存储在外部硬盘上,所以我去了iMac并开始在那里开展项目.我想我有两个不同版本的Xcode 10 beta,这可能是问题的一部分？基本上这条线在我的MacBook上工作正常,但在iMac上没有,btw正在运行macOS high Sierra 15:

guard let pixelBuffer : CVPixelBuffer = sampleBuffer.imageBuffer else { return }

它给了我以下错误:

Value of type 'CMSampleBuffer' has no member 'imageBuffer'

但是当我在我的macbook上打开完全相同的项目时,这没有任何问题.这可能是一个Xcode版本的东西,还是因为我运行两个不同的macOS版本？

我正在编写代码来解压本机附件 B H.264 流，我正在完成解析流的过程，从 SPS/PPS NALU 创建 CMVideoFormatDescription，并包装我从流中提取的其他 NALU在 CMSampleBuffers 中。

我在如何处理解码器的 CMBlockBuffer 和 CMSampleBuffer 内存方面遇到了心理障碍。我相信我的问题更多是缺乏对 CF 如何处理内存的透彻理解，所以我的问题更多的是关于这一点，但我希望上下文有帮助。

如果我像这样创建一个 CMBlockBuffer：

CMBlockBufferRef blockBuffer;

OSStatus status = CMBlockBufferCreateWithMemoryBlock(NULL,
                                                     memoryBlock,                       
                                                     blockBufferLength,
                                                     kCFAllocatorNull,
                                                     NULL,
                                                     0, 
                                                     blockBufferLength,          
                                                     kCMBlockBufferAlwaysCopyDataFlag | kCMBlockBufferAssureMemoryNowFlag,
                                                     &blockBuffer);

并将其添加到 CMSampleBuffer 中，如下所示：

CMSampleBufferRef sampleBuffer;

status = CMSampleBufferCreate(kCFAllocatorDefault,
                              blockBuffer,
                              true,
                              NULL,
                              NULL,
                              formatDescription,
                              1,
                              0,
                              NULL,
                              1,
                              &sampleSize,
                              &sampleBuffer);

我应该如何处理块缓冲区？SampleBuffer 是否保留块缓冲区的内存，或者我是否需要做一些事情来确保它没有被释放？

另外，关于异步解码过程，是否有一种明智的方法可以知道解码器何时使用 CMSampleBuffer 完成以便我可以处理它？

我的直觉告诉我 CMSampleBuffer 会保留 CMBlockBuffer，而 VTDecodeSession 会保留 CMSampleBuffer 直到它完成解码，但这是一个未记录的领域，我正在寻找一些方向。我得到的结果暗示我的直觉可能是错误的，所以我需要排除内存管理作为一个问题来保持我的理智......