我可以将CIFilter应用于ARkit相机Feed吗？

Question

我正在尝试将模糊效果应用于ARSCNView中的相机实时蒸汽图像.我查看了WWDC视频.他们只提到了Metal的自定义渲染,但我没有在网上找到任何完整的例子.知道怎么做吗？

---

更新1我试图将过滤器应用于后台.它显示不正确的方向.我怎样才能解决这个问题？

let bg=self.session.currentFrame?.capturedImage

        if(bg != nil){
            let context = CIContext()
            let filter:CIFilter=CIFilter(name:"CIColorInvert")!
            let image:CIImage=CIImage(cvPixelBuffer: bg!)
            filter.setValue(image, forKey: kCIInputImageKey)
            let result=filter.outputImage!
            self.sceneView.scene.background.contents = context.createCGImage(result, from: result.extent)

        }

Answer 1

background我找到了一个非常好的解决方案，只要设备方向发生变化，只需对属性应用相应的几何变换：

func session(_ session: ARSession, didUpdate frame: ARFrame) {
    let image = CIImage(cvPixelBuffer: frame.capturedImage)
    filter.setValue(image, forKey: kCIInputImageKey)

    let context = CIContext()
    if let result = filter.outputImage,
        let cgImage = context.createCGImage(result, from: result.extent) {

        sceneView.scene.background.contents = cgImage
        if let transform = currentScreenTransform() {
            sceneView.scene.background.contentsTransform = transform
        }
    }
}

private func currentScreenTransform() -> SCNMatrix4? {
    switch UIDevice.current.orientation {
    case .landscapeLeft:
        return SCNMatrix4Identity
    case .landscapeRight:
        return SCNMatrix4MakeRotation(.pi, 0, 0, 1)
    case .portrait:
        return SCNMatrix4MakeRotation(.pi / 2, 0, 0, 1)
    case .portraitUpsideDown:
        return SCNMatrix4MakeRotation(-.pi / 2, 0, 0, 1)
    default:
        return nil
    }
}

确保您首先调用UIDevice.current.beginGeneratingDeviceOrientationNotifications()了您的viewDidLoad方法。

Answer 2

我有同样的问题。你可以试试这个

let bg = sceneView.session.currentFrame?.capturedImage
    if (bg != nil) {
        let context = CIContext()
        let filter:CIFilter = CIFilter(name: "CIColorInvert")!

        let image:CIImage = CIImage(cvPixelBuffer: bg!, options: nil)

        let cgImage:CGImage = context.createCGImage(image, from: image.extent)!
        let uiImage:UIImage = UIImage.init(cgImage: cgImage)
        let resultImage = CIImage(image: uiImage)?.oriented(forExifOrientation: imageOrientationToDeviceOrientation(value: UIDevice.current.orientation))

        filter.setValue(resultImage, forKey: kCIInputImageKey)

        let result = filter.outputImage!
        self.sceneView.scene.background.contents = context.createCGImage(result, from: result.extent)
    }


func imageOrientationToDeviceOrientation(value: UIDeviceOrientation) -> Int32 {
            switch (value) {
            case .portrait:
                return 6
            case .landscapeLeft:
                return 1
            case .landscapeRight:
                return 3
            default:
                return 6
            }
        }