旋转视频而不旋转AVCaptureConnection并在AVAssetWriter会话中间

Question

我正在使用PBJVision来实现点击录制视频功能.图书馆不支持方向,所以我正在尝试设计它.从我看到的,旋转视频有三种方法 - 我需要帮助决定最佳前进方式以及如何实现它.请注意,可以在点击到记录段之间进行轮换.因此,在录制会话中,方向被锁定为用户点击按钮时的方向.下次用户点击按钮进行录制时,应将方向重新设置为设备的方向(因此生成的视频显示为正面朝上).

这些方法也在GitHub的问题页面中列出

方法1 旋转AVCaptureConnection使用setVideoOrientation:- 这会导致视频预览在每次切换时闪烁,因为这会切换实际的硬件.不酷,不可接受.

方法2transform在AVAssetWriterInput用于编写视频的对象上 设置属性.问题是,一旦资产编写者开始编写,该transform属性就无法更改,因此这仅适用于视频的第一部分.

方法3使用以下内容 旋转附加的图像缓冲区:如何直接在IOS 4中旋转CVImageBuffer图像而不转换为UIImage？但它一直在崩溃,我甚至不确定我是不是正在吠叫正确的树.有一个异常被抛出,我无法真正追溯到比我vImageRotate90_ARGB8888错误地使用该函数的事实.

关于我上面链接的GitHub问题页面的解释稍微详细一些.任何建议都会受到欢迎 - 说实话,我在AVFoundation上并不是很有经验,所以我希望有一些神奇的方法可以做到这一点,我甚至不知道!

Answer 1

根据Apple的文档,方法1不是首选方法("物理旋转缓冲区确实带有性能成本,因此只在必要时请求轮换").方法2适合我,但如果我在不支持转换"元数据"的应用上播放我的视频,则视频无法正常旋转.方法3就是我所做的.

我认为它崩溃了你之前,你正在试图将图像数据直接从传递vImageRotate...到AVAssetWriterInputPixelBufferAdaptor.你必须创建CVPixelBufferRef第一个.这是我的代码:

里面的captureOutput:didOutputSampleBuffer:fromConnection:我写它插入适配器前旋转框架:

if ([self.videoWriterInput isReadyForMoreMediaData])
{
    // Rotate buffer first and then write to adaptor
    CMTime sampleTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);
    CVPixelBufferRef rotatedBuffer = [self correctBufferOrientation:sampleBuffer];
    [self.videoWriterInputAdaptor appendPixelBuffer:rotatedBuffer withPresentationTime:sampleTime];
    CVBufferRelease(rotatedBuffer);
}

执行vImage旋转的引用函数是:

/* rotationConstant:
 *  0 -- rotate 0 degrees (simply copy the data from src to dest)
 *  1 -- rotate 90 degrees counterclockwise
 *  2 -- rotate 180 degress
 *  3 -- rotate 270 degrees counterclockwise
 */

- (CVPixelBufferRef)rotateBuffer:(CMSampleBufferRef)sampleBuffer withConstant:(uint8_t)rotationConstant
{
    CVImageBufferRef imageBuffer        = CMSampleBufferGetImageBuffer(sampleBuffer);
    CVPixelBufferLockBaseAddress(imageBuffer, 0);        

    OSType pixelFormatType              = CVPixelBufferGetPixelFormatType(imageBuffer);
    NSAssert(pixelFormatType == kCVPixelFormatType_32ARGB, @"Code works only with 32ARGB format. Test/adapt for other formats!");

    const size_t kAlignment_32ARGB      = 32;
    const size_t kBytesPerPixel_32ARGB  = 4;

    size_t bytesPerRow                  = CVPixelBufferGetBytesPerRow(imageBuffer);
    size_t width                        = CVPixelBufferGetWidth(imageBuffer);
    size_t height                       = CVPixelBufferGetHeight(imageBuffer);

    BOOL rotatePerpendicular            = (rotateDirection == 1) || (rotateDirection == 3); // Use enumeration values here
    const size_t outWidth               = rotatePerpendicular ? height : width;
    const size_t outHeight              = rotatePerpendicular ? width  : height;

    size_t bytesPerRowOut               = kBytesPerPixel_32ARGB * ceil(outWidth * 1.0 / kAlignment_32ARGB) * kAlignment_32ARGB;

    const size_t dstSize                = bytesPerRowOut * outHeight * sizeof(unsigned char);

    void *srcBuff                       = CVPixelBufferGetBaseAddress(imageBuffer);

    unsigned char *dstBuff              = (unsigned char *)malloc(dstSize);

    vImage_Buffer inbuff                = {srcBuff, height, width, bytesPerRow};
    vImage_Buffer outbuff               = {dstBuff, outHeight, outWidth, bytesPerRowOut};

    uint8_t bgColor[4]                  = {0, 0, 0, 0};

    vImage_Error err                    = vImageRotate90_ARGB8888(&inbuff, &outbuff, rotationConstant, bgColor, 0);
    if (err != kvImageNoError) 
    {
        NSLog(@"%ld", err);
    }

    CVPixelBufferUnlockBaseAddress(imageBuffer, 0);

    CVPixelBufferRef rotatedBuffer      = NULL;
    CVPixelBufferCreateWithBytes(NULL,
                                 outWidth,
                                 outHeight,
                                 pixelFormatType,
                                 outbuff.data,
                                 bytesPerRowOut,
                                 freePixelBufferDataAfterRelease,
                                 NULL,
                                 NULL,
                                 &rotatedBuffer);

    return rotatedBuffer;
}

void freePixelBufferDataAfterRelease(void *releaseRefCon, const void *baseAddress)
{
    // Free the memory we malloced for the vImage rotation
    free((void *)baseAddress);
}

注意:您可能希望使用枚举rotationConstant.类似的东西(不要调用此函数MOVRotateDirectionUnknown):

typedef NS_ENUM(uint8_t, MOVRotateDirection)
{
    MOVRotateDirectionNone = 0,
    MOVRotateDirectionCounterclockwise90,
    MOVRotateDirectionCounterclockwise180,
    MOVRotateDirectionCounterclockwise270,
    MOVRotateDirectionUnknown
};

注意:如果需要IOSurface支持,则应使用CVPixelBufferCreate而不是CVPixelBufferCreateWithBytes直接将字节数据传递到其中:

NSDictionary *pixelBufferAttributes = @{ (NSString *)kCVPixelBufferIOSurfacePropertiesKey : @{} };
CVPixelBufferCreate(kCFAllocatorDefault,
                    outWidth,
                    outHeight,
                    pixelFormatType,
                    (__bridge CFDictionaryRef)(pixelBufferAttributes),
                    &rotatedBuffer);

CVPixelBufferLockBaseAddress(rotatedBuffer, 0);
uint8_t *dest = CVPixelBufferGetBaseAddress(rotatedBuffer);
memcpy(dest, outbuff.data, bytesPerRowOut * outHeight);

CVPixelBufferUnlockBaseAddress(rotatedBuffer, 0);