测试失败时 XUnit 输出更多信息

Question

有谁知道当 XUnit 测试失败时如何向输出添加附加信息？我认为拥有它会是一件有用的事情。例如，我正在处理测试文件，我想看看哪一个失败了。

此问题包含有关使用 ITestOutputHelper 的信息，但这仅用于在成功运行测试期间打印信息。

Answer 1

失败的测试会引发异常。您可以捕获它们并使用 ITestOutputHelper 输出附加信息。

public class AuditCheck
{        
    public AuditCheck(ITestOutputHelper output)
    {
        this.output = output;
    }

    [Fact]
    public void MyTest()
    {
        var expected = 23;
        var actual = 42;
        try
        {
            Assert.Equal(expected, actual);
        }
        catch (XunitException e)
        {
            output.WriteLine($"{e.Message}: My own output, like filename, etc. Today is {DateTime.Today.DayOfWeek} and i expected {expected} but got {actual}");
            throw;
        }
        Assert.Equal(expected, actual);
    }
}

输出将写入日志，此外您可以通过单击 Visual Studio 测试资源管理器中的“输出”（捕获输出）来查看它：

Answer 2

如果您让 xUnit 将结果写入 xml 文件，例如，通过在命令行中告诉 xUnit 写入 XML 结果文件，您可以对 XML 文件的结果进行后处理，以获取有关失败的更多详细信息，只需稍加一点C# 代码（或插入您最喜欢的语言来代替 C#，例如 Python、Java 等。

虽然问题似乎是关于哪个测试用例失败，但对于数据驱动的测试或使用给定输入文件运行测试，写入 XML 结果文件的任何内容都是公平的，并且可用于此方法。因此，如果引发失败异常，并且您有测试用例标识可包含在异常中的信息，则应在 XML 结果文件中捕获该信息。

下面是在 C# 中执行此操作的一些代码示例。此代码的一个限制是它只需要 XML 中“assemblies”节点下的一个程序集。

public void ReadResultsXmlFile(string testResultsXmlFile)
{
    using (MyXmlTextReader = new XmlTextReader(testResultsXmlFile))
    {
        testResultXmlDocument = new XmlDocument();
        testResultXmlDocument.Load(MyXmlTextReader); // suppose that myXmlString contains "<Names>...</Names>"
        XmlNode xnAssembliesHeader = testResultXmlDocument.SelectSingleNode("/assemblies");
        TestRunDateTime = xnAssembliesHeader.Attributes.GetNamedItem("timestamp").Value;
        XmlNodeList xnAssemblyList = testResultXmlDocument.SelectNodes("/assemblies/assembly");
        foreach (XmlNode assembly in xnAssemblyList)
        {
            XmlNodeList xnTestList = testResultXmlDocument.SelectNodes(
                "/assemblies/assembly/collection/test");
            foreach (XmlNode test in xnTestList)
            {
                TestName = test.Attributes.GetNamedItem("name").Value;
                TestDuration = test.Attributes.GetNamedItem("time").Value;
                PassOrFail = test.Attributes.GetNamedItem("result").Value;
                // Failed test, get the failure information
                if (!(PassOrFail == "Pass"))
                {
                    Passed = false;
                    XmlNode failureData = test.SelectSingleNode("failure");
                    FailureText = failureData.InnerText;
                    StackTrace = failureData.SelectSingleNode("stack-trace").InnerText;
                    FailureMessage = failureData.SelectSingleNode("message").InnerText;
                    Console.WriteLine("Test: {0} Result: {1}  Error: {2}  StackTrace: {3}", TestName, PassOrFail, FailureMessage, FailureStackTrace);
                }
                else
                {
                    Passed = true;
                }
                Console.WriteLine("Test: {0} Result: {1}  Elapsed: {2}", TestName, PassOrFail, TestDuration);
            }
        }
    }
}

大约需要这些可用的声明，例如作为同一类的一部分：

public class MyTestAssembly
{
    public string TestName { get; set; }
    public string PassOrFail { get; set; }
    public bool Passed { get; set; }
    public string TestOutcome { get; set; }
    public string AlertLevel { get; set; }  // None, Warning, Alert
    public string AssemblyName { get; set; }
    public string AssemblyPath { get; set; }
    public string TestRunDateTime { get; set; }
    public TimeSpan testDuration;
    public string TestDuration { get; set; }
    public DateTime TestStartTime { get; set; }
    public DateTime TestStopTime { get; set; }
    public string TestResultsXmlFile { get; set; }
    public string FailureMessage { get; set; }
    public string StackTrace { get; set; }
    public string FailureStackTrace { get; set; }
    //todo: Report an error if more than one assembly exists under assemblies node
    public XmlDocument testResultXmlDocument;
    public string FailureText { get; set; }
    private XmlNodeList MyListOfTestAssembliesInResultsFile { get; set; }
    private XmlTextReader MyXmlTextReader;
    public double TestDurationSeconds;
    // And more stuff... this is only a partial example, extracted from working code.
}

我正在使用的 XUnit 结果文件来自 xUnit 2.4.x、.NET Framework，如下所示：

<?xml version="1.0" encoding="utf-8"?>
<assemblies timestamp="05/06/2019 19:06:38">
  <assembly name="C:\Program Files (x86)\RunXUnitTests\TestAssemblies\HRCafeORT.DLL" environment="64-bit .NET 4.0.30319.42000 [collection-per-class, parallel (4 threads)]" test-framework="xUnit.net 2.4.1.0" run-date="2019-05-06" run-time="19:06:30" config-file="C:\Program Files (x86)\RunXUnitTests\TestAssemblies\xunit.console.exe.Config" total="1" passed="0" failed="1" skipped="0" time="7.376" errors="0">
    <errors />
    <collection total="1" passed="0" failed="1" skipped="0" name="Test collection for HRCafeORT.HRCafeWebsiteORTFeature" time="6.814">
      <test name="Ensure Intranet Application HR Cafe is available and working properly." type="HRCafeORT.HRCafeWebsiteORTFeature" method="EnsureIntranetApplicationHRCafeIsAvailableAndWorkingProperly_" time="6.8143311" result="Fail">
        <traits>
          <trait name="FeatureTitle" value="HR Cafe Website ORT" />
          <trait name="Description" value="Ensure Intranet Application HR Cafe is available and working properly." />
          <trait name="Category" value="HRCafe" />
          <trait name="Category" value="IE" />
        </traits>
        <failure exception-type="OpenQA.Selenium.ElementNotInteractableException">
          <message><![CDATA[OpenQA.Selenium.ElementNotInteractableException : Element cannot be interacted with via the keyboard because it is not focusable]]></message>
          <stack-trace><![CDATA[   at OpenQA.Selenium.Remote.RemoteWebDriver.UnpackAndThrowOnError(Response errorResponse)
   at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
   at OpenQA.Selenium.Remote.RemoteWebElement.Execute(String commandToExecute, Dictionary`2 parameters)
   at HRCafeORT.HRCafeWebsiteORTSteps.GivenIHaveSelectedTheLeaveSelection() in C:\AppTest\ORT\HRCafeORT\HRCafeWebsiteORTSteps.cs:line 49
   at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStepMatch(BindingMatch match, Object[] arguments)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStep(IContextManager contextManager, StepInstance stepInstance)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnAfterLastStep()
   at TechTalk.SpecFlow.TestRunner.CollectScenarioErrors()
   at HRCafeORT.HRCafeWebsiteORTFeature.ScenarioCleanup()
   at HRCafeORT.HRCafeWebsiteORTFeature.EnsureIntranetApplicationHRCafeIsAvailableAndWorkingProperly_() in C:\AppTest\ORT\HRCafeORT\HRCafe.feature:line 14]]></stack-trace>
        </failure>
      </test>
    </collection>
  </assembly>
</assemblies>

控制台测试运行程序的典型 xUnitConsole 命令行如下所示。后跟输出文件名的 -xml 指定要在运行期间创建的 XML 结果文件。

"C:\RunXUnitTests\TestAssemblies\xunit.console.exe" "C:\RunXUnitTests\TestAssemblies\HRCafeORT.DLL"  -xml "C:\Users\Public\Documents\TestResults\HRCafeORT.xml"