使用System.Speech和Kinect

Question

我正在为大学项目开发​​一个原型演讲文本字幕应用程序.我将在我的项目中使用手势识别,所以我认为使用Kinect作为麦克风源是一个好主意,而不是使用额外的麦克风.我的应用的想法是识别自发的演讲,如长而复杂的句子(我明白,语音听写不会是完美的).我看过许多Kinect语音示例,它引用了Microsoft.Speech,但没有引用System.Speech.由于我需要训练语音引擎并将DictationGrammar加载到语音识别引擎中,因此Microsoft.Speech是我的唯一选择.

我已经设法使用Kinect作为直接麦克风音频源,但由于我正在加载Kinect进行视频预览和手势识别,我无法将其作为直接麦克风访问.

这是直接访问麦克风而无需加载Kinect硬件进行手势等的代码,并且工作正常:

private void InitializeSpeech()
{
    var speechRecognitionEngine = new SpeechRecognitionEngine();
    speechRecognitionEngine.SetInputToDefaultAudioDevice();
    speechRecognitionEngine.LoadGrammar(new DictationGrammar());
    speechRecognitionEngine.RecognizeAsync(RecognizeMode.Multiple);
    speechRecognitionEngine.SpeechRecognized += (s, args) => MessageBox.Show(args.Result.Text);
}

这是我需要在加载后通过Kinect访问访问源的地方,它根本没有做任何事情.我想这样做:

using (var audioSource = new KinectAudioSource())
{
    audioSource.FeatureMode = true;
    audioSource.AutomaticGainControl = false;
    audioSource.SystemMode = SystemMode.OptibeamArrayOnly;

    var recognizerInfo = GetKinectRecognizer();
    var speechRecognitionEngine = new SpeechRecognitionEngine(recognizerInfo.Id);

    speechRecognitionEngine.LoadGrammar(new DictationGrammar());
    speechRecognitionEngine.SpeechRecognized += (s, args) => MessageBox.Show(args.Result.Text);

    using (var s = audioSource.Start())
    {
        speechRecognitionEngine.SetInputToAudioStream(s, new SpeechAudioFormatInfo(EncodingFormat.Pcm, 16000, 16, 1, 32000, 2, null));
        speechRecognitionEngine.RecognizeAsync(RecognizeMode.Multiple);
    }
}

所以问题是,是否可以使用System.Speech而不是Microsoft.Speech与当前的Kinect SDK,以及我在第二个代码示例中做错了什么？

GetKinectRecognizer方法

private static RecognizerInfo GetKinectRecognizer()
{
    Func<RecognizerInfo, bool> matchingFunc = r =>
    {
        string value;
        r.AdditionalInfo.TryGetValue("Kinect", out value);
        return "True".Equals(value, StringComparison.InvariantCultureIgnoreCase) && "en-US".Equals(r.Culture.Name, StringComparison.InvariantCultureIgnoreCase);
    };

    return SpeechRecognitionEngine.InstalledRecognizers().Where(matchingFunc).FirstOrDefault();
}

Answer 1

根据我自己的实验，我可以告诉您实际上可以同时使用这两个库。

尝试使用此代码而不是当前代码（显然，请确保添加对 System.Speech 的引用）：

using (var audioSource = new KinectAudioSource())
{
    audioSource.FeatureMode = true;
    audioSource.AutomaticGainControl = false;
    audioSource.SystemMode = SystemMode.OptibeamArrayOnly;

    System.Speech.Recognition.RecognizerInfo ri = GetKinectRecognizer();
    var speechRecognitionEngine = new SpeechRecognitionEngine(ri.Id);

    speechRecognitionEngine.LoadGrammar(new DictationGrammar());
    speechRecognitionEngine.SpeechRecognized += (s, args) => MessageBox.Show(args.Result.Text);

    using (var s = audioSource.Start())
    {
        speechRecognitionEngine.SetInputToAudioStream(s, new SpeechAudioFormatInfo(EncodingFormat.Pcm, 16000, 16, 1, 32000, 2, null));
        speechRecognitionEngine.RecognizeAsync(RecognizeMode.Multiple);
    }
}

祝你好运！！！