VNTrackRectangleRequest内部错误

Question

我想要一个简单的矩形跟踪控制器,我可以得到矩形检测正常,但跟踪请求总是因为我找不到的原因而失败.

有时跟踪请求会在失败之前触发它几次回调,有时它会在单次回调发生之前立即失败.我觉得这与我提交请求的方式有关,但我无法深究它.

这是视图控制器的代码

class TestController: UIViewController, AVCaptureVideoDataOutputSampleBufferDelegate {

    // Video capture
    private var videoSession = AVCaptureSession()
    private var videoLayer: AVCaptureVideoPreviewLayer!

    // Detection
    private var detectionRequest: VNDetectRectanglesRequest?
    private let sequenceHandler = VNSequenceRequestHandler()

    // Tracking
    private var trackingRequest: VNTrackRectangleRequest?
    private var shape: Detection?
    private var pixelBuffer: CVImageBuffer?




    // MARK: Setup
    override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)

        startVideoFeed()
    }

    override func viewDidLayoutSubviews() {
        videoLayer.frame = view.layer.bounds
    }

    override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
        startDetectingRectangles()
    }



    private func startDetectingRectangles() {
        let request = VNDetectRectanglesRequest(completionHandler: didDetectRectangle)
        request.maximumObservations = 1
        request.minimumSize = 0.07
        request.minimumConfidence = 0.9
        request.minimumAspectRatio = 0.5
        request.maximumAspectRatio = 2
        request.quadratureTolerance = 10
        detectionRequest = request
    }

    private func didDetectRectangle(request: VNRequest, error: Error?) {

        // Fetch results of the correct type
        guard let observations = request.results, observations.count > 0 else { return }
        let results = observations.map { $0 as? VNRectangleObservation }

        for case let rectangle? in results {
            detectionRequest = nil

            let request = VNTrackRectangleRequest(rectangleObservation: rectangle, completionHandler: didTrackRectangle)
            trackingRequest = request
        }
    }


    private func didTrackRectangle(request: VNRequest, error: Error?) {

        // Fetch results of the correct type
        guard let observation = request.results?.first as? VNRectangleObservation else { return }

        // Create or update UI
    }




    // Start capturing video frames
    private func startVideoFeed() {

        // Session config
        videoSession.sessionPreset = .photo

        // Create device and input to device
        guard
            let captureDevice = AVCaptureDevice.default(for: .video),
            let deviceInput = try? AVCaptureDeviceInput(device: captureDevice)
            else {
                fatalError("Error setting up capture device.")
        }

        // Setup device output
        let deviceOutput = AVCaptureVideoDataOutput()
        deviceOutput.setSampleBufferDelegate(self, queue: DispatchQueue.global(qos: .default))

        // Set input and output
        videoSession.addInput(deviceInput)
        videoSession.addOutput(deviceOutput)

        // Setup video display layer
        videoLayer = AVCaptureVideoPreviewLayer(session: videoSession)
        videoLayer.frame = view.bounds
        videoLayer.videoGravity = .resizeAspectFill
        view.layer.addSublayer(videoLayer)

        videoSession.startRunning()
    }

    func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {

        pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)

        guard let pixelBuffer = pixelBuffer else { return }
        do {

            var requests: [VNRequest] = []

            if let detectionRequest = detectionRequest {
                requests.append(detectionRequest) }
            if let trackingRequest = trackingRequest {
                requests.append(trackingRequest) }

            try sequenceHandler.perform(requests, on: pixelBuffer, orientation: .right)
        } catch {
            print(error)
        }
    }
}

错误是打印:

错误域= com.apple.vis代码= 9"内部错误:跟踪其中一个角失败,置信度= 0.000000;阈值= 0.650000"UserInfo = {NSLocalizedDescription =内部错误:跟踪其中一个角失败,置信度= 0.000000 ; 阈值= 0.650000}

在跟踪请求首次失败后每帧输出一次,并且请求永远不会恢复.它try sequenceHandler.perform(requests, on: pixelBuffer, orientation: .right)是抛出错误的行(然后我打印).

我查看了网络上点缀的对象观察示例,他们似乎需要在每个帧上进行新的跟踪请求以跟踪对象.我已经尝试VNTrackRectangleRequest在didTrackRectangle函数中创建一个新的,所以每个框架都有一个新请求,但我遇到了同样的问题.

对此有任何帮助,我们非常感谢.

Answer 1

我能够毫无问题地运行您的代码，至少在跟踪清晰可见的矩形时是这样。

VNSequenceRequestHandler当跟踪请求失败时，您可能应该创建一个新的。似乎当跟踪请求失去对对象的跟踪时间太长时，它将无法恢复并再次开始跟踪它。这也意味着它们永远不会被释放，并且会不断抛出错误。如果您继续向请求处理程序添加新的跟踪请求，您很快就会超出同时跟踪请求的限制。

如果您能够预测跟踪序列的结束，则可以将跟踪请求的isLastFrametrue属性设置为，当当前帧完成处理时，它将被释放到可用跟踪器池中。