在 iOS 中实时访问视频帧

Question

我必须从服务器访问视频，我希望能够实时访问它的帧，以便我可以对它们进行一些图像处理。

我知道如何使用 AVAssetReader 离线完成，但此解决方案不适用于实时处理。

Answer 1

您可以使用实时访问远程资产的帧AVPlayerItemVideoOutput（感谢objc.io）。

ps 始终检查您的远程资产是否被App Transport Security阻止。

ppsAVAssetReader对远程资产的支持很复杂。

import UIKit
import AVFoundation

class ViewController: UIViewController {        
    let player = AVPlayer(url: URL(string: "http://0.s3.envato.com/h264-video-previews/80fad324-9db4-11e3-bf3d-0050569255a8/490527.mp4")!)
    let videoOutput = AVPlayerItemVideoOutput(pixelBufferAttributes: [String(kCVPixelBufferPixelFormatTypeKey): NSNumber(value: kCVPixelFormatType_32BGRA)])

    override func viewDidLoad() {
        super.viewDidLoad()

        player.currentItem!.add(videoOutput)
        player.play()

        let displayLink = CADisplayLink(target: self, selector: #selector(displayLinkDidRefresh(link:)))
        displayLink.add(to: RunLoop.main, forMode: RunLoopMode.commonModes)
    }

    func displayLinkDidRefresh(link: CADisplayLink) {
        let itemTime = videoOutput.itemTime(forHostTime: CACurrentMediaTime())

        if videoOutput.hasNewPixelBuffer(forItemTime: itemTime) {
            if let pixelBuffer = videoOutput.copyPixelBuffer(forItemTime: itemTime, itemTimeForDisplay: nil) {
                print("pixelBuffer \(pixelBuffer)") // yay, pixel buffer
                let image = CIImage(cvImageBuffer: pixelBuffer) // or maybe CIImage?
                print("CIImage \(image)")
            }
        }
    }
}