iOS 10.0 - 10.1:使用AVVideoCompositionCoreAnimationTool后,AVPlayerLayer不显示视频

Question

如果您想自己运行,这是一个完整的项目:https://www.dropbox.com/s/5p384mogjzflvqk/AVPlayerLayerSoundOnlyBug_iOS10.zip?dl=0

这是iOS 10上的一个新问题,从iOS 10.2开始修复.在导出期间使用AVAssetExportSession和AVVideoCompositionCoreAnimationTool导出视频以在视频顶部合成图层后,AVPlayerLayer中播放的视频无法播放.这似乎不是由于达到AV编码/解码流水线限制造成的,因为它经常在单次导出后发生,据我所知只会旋转2个流水线:1为AVAssetExportSession而另一个为AVPlayer.我也正确设置了图层的框架,正如您可以通过运行下面的代码看到的那样,该图层为图层提供了一个您可以清楚看到的蓝色背景.

在导出之后,在播放视频之前等待一段时间似乎使其更加可靠,但这并不是一个可以接受的解决方法来告诉用户.

有关导致此问题或我如何解决或解决问题的任何想法？我搞砸了什么或者错过了一个重要的步骤或细节？任何帮助或文档指针都非常感谢.

import UIKit
import AVFoundation

/* After exporting an AVAsset using AVAssetExportSession with AVVideoCompositionCoreAnimationTool, we
 * will attempt to play a video using an AVPlayerLayer with a blue background.
 *
 * If you see the blue background and hear audio you're experiencing the missing-video bug. Otherwise
 * try hitting the button again.
 */

class ViewController: UIViewController {
    private var playerLayer: AVPlayerLayer?
    private let button = UIButton()
    private let indicator = UIActivityIndicatorView(activityIndicatorStyle: .gray)

    override func viewDidLoad() {
        super.viewDidLoad()
        view.backgroundColor = UIColor.white
        button.setTitle("Cause Trouble", for: .normal)
        button.setTitleColor(UIColor.black, for: .normal)
        button.addTarget(self, action: #selector(ViewController.buttonTapped), for: .touchUpInside)
        view.addSubview(button)
        button.translatesAutoresizingMaskIntoConstraints = false
        NSLayoutConstraint.activate([
            button.centerXAnchor.constraint(equalTo: view.centerXAnchor),
            button.bottomAnchor.constraint(equalTo: view.bottomAnchor, constant: -16),
        ])

        indicator.hidesWhenStopped = true
        view.insertSubview(indicator, belowSubview: button)
        indicator.translatesAutoresizingMaskIntoConstraints = false
        NSLayoutConstraint.activate([
            indicator.centerXAnchor.constraint(equalTo: button.centerXAnchor),
            indicator.centerYAnchor.constraint(equalTo: button.centerYAnchor),
        ])
    }

    func buttonTapped() {
        button.isHidden = true
        indicator.startAnimating()
        playerLayer?.removeFromSuperlayer()

        let sourcePath = Bundle.main.path(forResource: "video.mov", ofType: nil)!
        let sourceURL = URL(fileURLWithPath: sourcePath)
        let sourceAsset = AVURLAsset(url: sourceURL)

        //////////////////////////////////////////////////////////////////////
        // STEP 1: Export a video using AVVideoCompositionCoreAnimationTool //
        //////////////////////////////////////////////////////////////////////
        let exportSession = { () -> AVAssetExportSession in
            let sourceTrack = sourceAsset.tracks(withMediaType: AVMediaTypeVideo).first!

            let parentLayer = CALayer()
            parentLayer.frame = CGRect(origin: .zero, size: CGSize(width: 1280, height: 720))
            let videoLayer = CALayer()
            videoLayer.frame = parentLayer.bounds
            parentLayer.addSublayer(videoLayer)

            let composition = AVMutableVideoComposition(propertiesOf: sourceAsset)
            composition.animationTool = AVVideoCompositionCoreAnimationTool(postProcessingAsVideoLayer: videoLayer, in: parentLayer)
            let layerInstruction = AVMutableVideoCompositionLayerInstruction(assetTrack: sourceTrack)
            layerInstruction.setTransform(sourceTrack.preferredTransform, at: kCMTimeZero)
            let instruction = AVMutableVideoCompositionInstruction()
            instruction.timeRange = CMTimeRange(start: kCMTimeZero, duration: sourceAsset.duration)
            instruction.layerInstructions = [layerInstruction]
            composition.instructions = [instruction]

            let e = AVAssetExportSession(asset: sourceAsset, presetName: AVAssetExportPreset1280x720)!
            e.videoComposition = composition
            e.outputFileType = AVFileTypeQuickTimeMovie
            e.timeRange = CMTimeRange(start: kCMTimeZero, duration: sourceAsset.duration)
            let outputURL = URL(fileURLWithPath: NSTemporaryDirectory().appending("/out2.mov"))
            _ = try? FileManager.default.removeItem(at: outputURL)
            e.outputURL = outputURL
            return e
        }()

        print("Exporting asset...")
        exportSession.exportAsynchronously {
            assert(exportSession.status == .completed)

            //////////////////////////////////////////////
            // STEP 2: Play a video in an AVPlayerLayer //
            //////////////////////////////////////////////
            DispatchQueue.main.async {
                // Reuse player layer, shouldn't be hitting the AV pipeline limit
                let playerItem = AVPlayerItem(asset: sourceAsset)
                let layer = self.playerLayer ?? AVPlayerLayer()
                if layer.player == nil {
                    layer.player = AVPlayer(playerItem: playerItem)
                }
                else {
                    layer.player?.replaceCurrentItem(with: playerItem)
                }
                layer.backgroundColor = UIColor.blue.cgColor
                if UIDeviceOrientationIsPortrait(UIDevice.current.orientation) {
                    layer.frame = self.view.bounds
                    layer.bounds.size.height = layer.bounds.width * 9.0 / 16.0
                }
                else {
                    layer.frame = self.view.bounds.insetBy(dx: 0, dy: 60)
                    layer.bounds.size.width = layer.bounds.height * 16.0 / 9.0
                }
                self.view.layer.insertSublayer(layer, at: 0)
                self.playerLayer = layer

                layer.player?.play()
                print("Playing a video in an AVPlayerLayer...")

                self.button.isHidden = false
                self.indicator.stopAnimating()
            }
        }
    }
}

Answer 1

在这种情况下,我的答案是AVVideoCompositionCoreAnimationTool通过使用实现AVVideoCompositing协议的自定义视频合成类以及实现协议的自定义组合指令来解决该问题AVVideoCompositionInstruction.因为我需要覆盖CALayer视频的顶部,我在合成指令实例中包含该层.

您需要在视频合成中设置自定义合成器,如下所示:

composition.customVideoCompositorClass = CustomVideoCompositor.self

然后在其上设置自定义说明:

let instruction = CustomVideoCompositionInstruction(...) // whatever parameters you need and are required by the instruction protocol
composition.instructions = [instruction]

编辑:这是一个如何使用自定义合成器使用GPU覆盖视频图层的工作示例:https://github.com/samsonjs/LayerVideoCompositor ...原始答案在下面继续

至于合成器本身,如果您观看相关的WWDC会话并查看其示例代码,则可以实现一个.我不能发布一个我写到这里,但我使用CoreImage做在处理繁重AVAsynchronousVideoCompositionRequest,确保使用最佳性能的OpenGL CoreImage上下文(如果你这样做在CPU上这将是一败涂地缓慢).如果在导出期间出现内存使用量峰值,您可能还需要一个自动释放池.

如果你CALayer像我一样覆盖,那么一定要layer.isGeometryFlipped = true在将该层渲染到a CGImage之前进行设置,然后再将其发送到CoreImage.并确保CGImage在合成器中缓存帧与帧之间的渲染.