将PCM(CMSampleBufferRef)编码到iOS上的AAC - 如何设置频率和比特率？

Question

我想将PCM(CMSampleBufferRef(s)上线)编码AVCaptureAudioDataOutputSampleBufferDelegate到AAC中.

当第一个CMSampleBufferRef到达时,我AudioStreamBasicDescription根据文档设置(进/出)(s),"out"

AudioStreamBasicDescription inAudioStreamBasicDescription = *CMAudioFormatDescriptionGetStreamBasicDescription((CMAudioFormatDescriptionRef)CMSampleBufferGetFormatDescription(sampleBuffer));

AudioStreamBasicDescription outAudioStreamBasicDescription = {0}; // Always initialize the fields of a new audio stream basic description structure to zero, as shown here: ...
outAudioStreamBasicDescription.mSampleRate = 44100; // The number of frames per second of the data in the stream, when the stream is played at normal speed. For compressed formats, this field indicates the number of frames per second of equivalent decompressed data. The mSampleRate field must be nonzero, except when this structure is used in a listing of supported formats (see “kAudioStreamAnyRate”).
outAudioStreamBasicDescription.mFormatID = kAudioFormatMPEG4AAC; // kAudioFormatMPEG4AAC_HE does not work. Can't find `AudioClassDescription`. `mFormatFlags` is set to 0.
outAudioStreamBasicDescription.mFormatFlags = kMPEG4Object_AAC_SSR; // Format-specific flags to specify details of the format. Set to 0 to indicate no format flags. See “Audio Data Format Identifiers” for the flags that apply to each format.
outAudioStreamBasicDescription.mBytesPerPacket = 0; // The number of bytes in a packet of audio data. To indicate variable packet size, set this field to 0. For a format that uses variable packet size, specify the size of each packet using an AudioStreamPacketDescription structure.
outAudioStreamBasicDescription.mFramesPerPacket = 1024; // The number of frames in a packet of audio data. For uncompressed audio, the value is 1. For variable bit-rate formats, the value is a larger fixed number, such as 1024 for AAC. For formats with a variable number of frames per packet, such as Ogg Vorbis, set this field to 0.
outAudioStreamBasicDescription.mBytesPerFrame = 0; // The number of bytes from the start of one frame to the start of the next frame in an audio buffer. Set this field to 0 for compressed formats. ...
outAudioStreamBasicDescription.mChannelsPerFrame = 1; // The number of channels in each frame of audio data. This value must be nonzero.
outAudioStreamBasicDescription.mBitsPerChannel = 0; // ... Set this field to 0 for compressed formats.
outAudioStreamBasicDescription.mReserved = 0; // Pads the structure out to force an even 8-byte alignment. Must be set to 0.

和AudioConverterRef.

AudioClassDescription audioClassDescription;
memset(&audioClassDescription, 0, sizeof(audioClassDescription));
UInt32 size;
NSAssert(AudioFormatGetPropertyInfo(kAudioFormatProperty_Encoders, sizeof(outAudioStreamBasicDescription.mFormatID), &outAudioStreamBasicDescription.mFormatID, &size) == noErr, nil);
uint32_t count = size / sizeof(AudioClassDescription);
AudioClassDescription descriptions[count];
NSAssert(AudioFormatGetProperty(kAudioFormatProperty_Encoders, sizeof(outAudioStreamBasicDescription.mFormatID), &outAudioStreamBasicDescription.mFormatID, &size, descriptions) == noErr, nil);
for (uint32_t i = 0; i < count; i++) {

    if ((outAudioStreamBasicDescription.mFormatID == descriptions[i].mSubType) && (kAppleSoftwareAudioCodecManufacturer == descriptions[i].mManufacturer)) {

        memcpy(&audioClassDescription, &descriptions[i], sizeof(audioClassDescription));

    }
}
NSAssert(audioClassDescription.mSubType == outAudioStreamBasicDescription.mFormatID && audioClassDescription.mManufacturer == kAppleSoftwareAudioCodecManufacturer, nil);
AudioConverterRef audioConverter;
memset(&audioConverter, 0, sizeof(audioConverter));
NSAssert(AudioConverterNewSpecific(&inAudioStreamBasicDescription, &outAudioStreamBasicDescription, 1, &audioClassDescription, &audioConverter) == 0, nil);

然后,我将每个转换CMSampleBufferRef为原始AAC数据.

AudioBufferList inAaudioBufferList;
CMBlockBufferRef blockBuffer;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBuffer, NULL, &inAaudioBufferList, sizeof(inAaudioBufferList), NULL, NULL, 0, &blockBuffer);
NSAssert(inAaudioBufferList.mNumberBuffers == 1, nil);

uint32_t bufferSize = inAaudioBufferList.mBuffers[0].mDataByteSize;
uint8_t *buffer = (uint8_t *)malloc(bufferSize);
memset(buffer, 0, bufferSize);
AudioBufferList outAudioBufferList;
outAudioBufferList.mNumberBuffers = 1;
outAudioBufferList.mBuffers[0].mNumberChannels = inAaudioBufferList.mBuffers[0].mNumberChannels;
outAudioBufferList.mBuffers[0].mDataByteSize = bufferSize;
outAudioBufferList.mBuffers[0].mData = buffer;

UInt32 ioOutputDataPacketSize = 1;

NSAssert(AudioConverterFillComplexBuffer(audioConverter, inInputDataProc, &inAaudioBufferList, &ioOutputDataPacketSize, &outAudioBufferList, NULL) == 0, nil);

NSData *data = [NSData dataWithBytes:outAudioBufferList.mBuffers[0].mData length:outAudioBufferList.mBuffers[0].mDataByteSize];

free(buffer);
CFRelease(blockBuffer);

inInputDataProc() 执行:

OSStatus inInputDataProc(AudioConverterRef inAudioConverter, UInt32 *ioNumberDataPackets, AudioBufferList *ioData, AudioStreamPacketDescription **outDataPacketDescription, void *inUserData)
{
    AudioBufferList audioBufferList = *(AudioBufferList *)inUserData;

    ioData->mBuffers[0].mData = audioBufferList.mBuffers[0].mData;
    ioData->mBuffers[0].mDataByteSize = audioBufferList.mBuffers[0].mDataByteSize;

    return  noErr;
}

现在,data保存我的原始AAC,我用适当的ADTS头包装到ADTS帧中,这些ADTS帧的序列是可播放的AAC文件.

但我并不像我想的那样理解这段代码.一般来说,我不明白音频...我刚刚在博客,论坛和文档之后以某种方式写了它,现在它已经有效但我不知道为什么以及如何更改一些参数.所以这是我的问题:

1. 我需要在HW编码器占用(by AVAssetWriter)期间使用此转换器.这就是我通过SW AudioConverterNewSpecific()而不是SW转换器的原因AudioConverterNew().但现在设置outAudioStreamBasicDescription.mFormatID = kAudioFormatMPEG4AAC_HE;不起作用.找不到AudioClassDescription.即使mFormatFlags设置为0.使用kAudioFormatMPEG4AAC(kMPEG4Object_AAC_SSR)结束时我失去了kAudioFormatMPEG4AAC_HE什么？我应该将什么用于直播？kMPEG4Object_AAC_SSR还是kMPEG4Object_AAC_Main？

2. 如何正确改变采样率？outAudioStreamBasicDescription.mSampleRate例如,如果我设置为22050或8000,则音频播放速度就会变慢.我将ADTS标头中的采样频率索引设置为相同的频率outAudioStreamBasicDescription.mSampleRate.

3. 如何改变比特率？ffmpeg -i显示了生成aac的这个信息: Stream #0:0: Audio: aac, 44100 Hz, mono, fltp, 64 kb/s.例如,如何将其更改为16 kbps？随着我降低频率,比特率正在下降,但我相信这不是唯一的方法吗？然而,正如我在2中提到的那样,通过降低频率来损坏播放.

4. 如何计算大小buffer？现在我把它设置为,uint32_t bufferSize = inAaudioBufferList.mBuffers[0].mDataByteSize;因为我相信压缩格式不会比未压缩格式大......但不是不必要地太多了吗？

5. 如何ioOutputDataPacketSize正确设置？如果我正确获得文档,我应该将其设置为0. UInt32 ioOutputDataPacketSize = bufferSize / outAudioStreamBasicDescription.mBytesPerPacket;但mBytesPerPacket如果我将其设置为0,则AudioConverterFillComplexBuffer()返回错误.如果我将它设置为1,它可以工作,但我不知道为什么...

6. 在inInputDataProc()那里有3个"out"参数.我刚刚开始ioData.如果我还设置ioNumberDataPackets及outDataPacketDescription？为什么以及如何？

Answer 1

在将音频输入 AAC 转换器之前，您可能需要使用重采样音频单元更改原始音频数据的采样率。否则 AAC 标头和音频数据之间将会不匹配。