AVAssetImageGenerator有时返回来自2个连续帧的相同图像

Question

我正在从视频中提取每一帧AVAssetImageGenerator,但有时它会连续两次返回我几乎相同的图像(它们没有相同的"帧时间").有趣的是它总是发生(在我的测试视频中)每5帧.

这里和这里是两个图像(在新选项卡中打开每个图像然后切换选项卡以查看差异).

这是我的代码:

//setting up generator & compositor
self.generator = [AVAssetImageGenerator assetImageGeneratorWithAsset:asset];
generator.appliesPreferredTrackTransform = YES;
self.composition = [AVVideoComposition videoCompositionWithPropertiesOfAsset:asset];

NSTimeInterval duration = CMTimeGetSeconds(asset.duration);
NSTimeInterval frameDuration = CMTimeGetSeconds(composition.frameDuration);
CGFloat totalFrames = round(duration/frameDuration);

NSMutableArray * times = [NSMutableArray array];
for (int i=0; i<totalFrames; i++) {
    NSValue * time = [NSValue valueWithCMTime:CMTimeMakeWithSeconds(i*frameDuration, composition.frameDuration.timescale)];
    [times addObject:time];
}

AVAssetImageGeneratorCompletionHandler handler = ^(CMTime requestedTime, CGImageRef im, CMTime actualTime, AVAssetImageGeneratorResult result, NSError *error){
    // If actualTime is not equal to requestedTime image is ignored
    if(CMTimeCompare(actualTime, requestedTime) == 0) {
        if (result == AVAssetImageGeneratorSucceeded) {
            NSLog(@"%.02f     %.02f", CMTimeGetSeconds(requestedTime), CMTimeGetSeconds(actualTime));
            // Each log have differents actualTimes.
            // frame extraction is here...
        }
    }
};

generator.requestedTimeToleranceBefore = kCMTimeZero;
generator.requestedTimeToleranceAfter = kCMTimeZero;
[generator generateCGImagesAsynchronouslyForTimes:times completionHandler:handler];

知道它可能来自哪里？

Answer 1

请参阅AVAssetImageGenerator的以下属性.您应该为两个属性设置kCMTimeZero以获取确切的帧.

/* The actual time of the generated images will be within the range [requestedTime-toleranceBefore, requestedTime+toleranceAfter] and may differ from the requested time for efficiency.
   Pass kCMTimeZero for both toleranceBefore and toleranceAfter to request frame-accurate image generation; this may incur additional decoding delay.
   Default is kCMTimePositiveInfinity. */
@property (nonatomic) CMTime requestedTimeToleranceBefore NS_AVAILABLE(10_7, 5_0);
@property (nonatomic) CMTime requestedTimeToleranceAfter NS_AVAILABLE(10_7, 5_0);

在为两个属性设置kCMTimeZero之前,我为您提供了与您所经历的不同请求时间相同的图像.只需尝试以下代码即可.

self.imageGenerator = [AVAssetImageGenerator assetImageGeneratorWithAsset:myAsset];
self.imageGenerator.requestedTimeToleranceBefore = kCMTimeZero;
self.imageGenerator.requestedTimeToleranceAfter = kCMTimeZero;

Answer 2

我使用了一种稍微不同的方法来计算 CMTime 请求，它似乎有效。这是代码（假设是 iOS）：

-(void)extractImagesFromMovie {

// set the asset
    NSString* path = [[NSBundle mainBundle] pathForResource:@"myMovie" ofType:@"MOV"];
    NSURL* movURL = [NSURL fileURLWithPath:path];

NSMutableDictionary* myDict = [NSMutableDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithBool:YES] , 
                                 AVURLAssetPreferPreciseDurationAndTimingKey , 
                                [NSNumber numberWithInt:0],
                                AVURLAssetReferenceRestrictionsKey, nil];

AVURLAsset* movie = [[AVURLAsset alloc] initWithURL:movURL options:myDict];


// set the generator
AVAssetImageGenerator* generator = [[AVAssetImageGenerator assetImageGeneratorWithAsset:movie] retain];
generator.requestedTimeToleranceBefore = kCMTimeZero;
generator.requestedTimeToleranceAfter = kCMTimeZero;

// look for the video track
AVAssetTrack* videoTrack;
bool foundTrack = NO;

for (AVAssetTrack* track in movie.tracks) {

    if ([track.mediaType isEqualToString:@"vide"]) {
        if (foundTrack) {NSLog (@"Error - - - more than one video tracks"); return(-1);}
        else {
            videoTrack = track;
            foundTrack = YES;
        }
    }
}
if (foundTrack == NO) {NSLog (@"Error - - No Video Tracks at all"); return(-1);}

// set the number of frames in the movie
int frameRate = videoTrack.nominalFrameRate;
float value = movie.duration.value;
float timeScale = movie.duration.timescale;
float totalSeconds = value / timeScale;
int totalFrames = totalSeconds * frameRate;

NSLog (@"total frames %d", totalFrames);

int timeValuePerFrame = movie.duration.timescale / frameRate;

NSMutableArray* allFrames = [[NSMutableArray new] retain];

// get each frame
for (int k=0; k< totalFrames; k++) {

    int timeValue = timeValuePerFrame * k;
    CMTime frameTime;
    frameTime.value = timeValue;
    frameTime.timescale = movie.duration.timescale;
    frameTime.flags = movie.duration.flags;
    frameTime.epoch = movie.duration.epoch;

    CMTime gotTime;

    CGImageRef myRef = [generator copyCGImageAtTime:frameTime actualTime:&gotTime error:nil];
    [allFrames addObject:[UIImage imageWithCGImage:myRef]];

    if (gotTime.value != frameTime.value) NSLog (@"requested %lld got %lld for k %d", frameTime.value, gotTime.value, k)

}

NSLog (@"got %d images in the array", [allFrames count]);
// do something with images here...
}